Re: [egit-dev] How to build [EJ]Git?

[Jason van Zyl <jason@xxxxxxxxx>]

On 2010-01-07, at 2:51 AM, Thomas Hallgren wrote:

> On 2010-01-07 08:09, Gunnar Wagenknecht wrote:
>> Am 06.01.2010 23:47, schrieb Jason van Zyl:
>>   
>>> But insofar as being locked out JGit using Maven means that folks
>>> using IDEA, Netbeans and Eclipse can hack on JGit with a great degree of
>>> support. If you used PDE Build, for example you would immediately
>>> lockout IDEA and Netbeans users. At least I don't think there is any PDE
>>> Build support in IDEA and Netbeans, correct me if I'm wrong.
>>>     
>> I don't think so. In order to hack on JGit I souldn't need Maven, PDE
>> Build or Buckminster (btw, PDE Build != PDE Tooling). I just need an
>> editor and a Java compiler. Most IDEs come with both. One IDE might have
>> better support for setting up the classpath but that's an IDE specific
>> part which should not influence how other IDE users are used to work in
>> their environment.
>> 
>>   
> I concur with that. Any IDE should be able to just check things out and build them. OSGi compliant manifests should always be checked in as a part of the code base.

That's just not the way it works right now, and BND has been around for a long time and it's never worked that way for BND. There are probably more OSGi projects using the maven-bundle-plugin and they use BND files. The builds at the OSGi Alliance use BND so they do some manifest generation. So many people outside of Eclipse use OSGi in conjunction with BND and having the actual MANIFEST.MF in complete form is just not the way it works in the day-to-day development of many people attempting to use OSGi.

You can't just ignore this mode of development. We are trying to accommodate both and make them work together. There is simply no other workable situation. There are already  many fractured efforts and the only way to have this work in the future is to mediate at 1) project level where we have manifest-first and pom-first (and really just BND file first) as well as at the repository level and 2) the repository level where we have Maven, P2, OBR which are all in use these days.

> They are not IDE specific at all and should be considered an integral part of the bundle declaring things like bundle activation and downstream package visibility.
> 

That's just simply not the reality of todays development with OSGi. Especially with very popular technologies like Spring DM, and Peaberry where modularity is encouraged but there are other representations of the manifest, or they are hidden all together. OSGi itself separates the manifest from the artifact at fundamental level so you can expect people to keep taking advantage of this fact.

> When it comes to IDE specific meta-data, I think that refraning from checking that in, just because it doesn't make sense to other IDE's, is severly limiting.

You've got people in both camps there. There are lots of teams that don't like project specific files, and depending on the integration complete control over the target environment is possible. With a pom.xml for example a project works in IDEA, Netbeans and Eclipse. Netbeans has ways of controlling the IDE specifically from the POM, and m2eclipse has configurators which can bind directly into Eclipse with builders and APIs like JDT. It's a balance but we're finding that approach working an enterprises especially like it with M2Eclipse because the configurators we have are bi-directional: change the pom.xml and the Eclipse preferences are modified accordingly. Change the Eclipse settings and the pom.xml is updated accordingly. The problem that we found in organizations with checking in project specific metadata was that the automated builds would not reflect what developers changed in the IDE and you end up with the "it works in my backyard" situation. There are over 300 maven plugins, and there are matching Eclipse plugins in almost all cases and people want these to work together. The configurators in M2Eclipse are what bind these together so we can leverage both sides of the equation. 

This is exactly what we're going to do with the maven-bundle-plugin and PDE and what we did the Tycho and PDE. A configurator will mediate between the Maven side and the Eclipse side to make sure the automated build works, and that the developer experience is good. We can leverage any incremental techniques necessary to make things fast in the IDE. 

With Tycho we use as much of the PDE information as we can and that works most of the time. There are cases where we simply have to perform some configuration in the POM. For example building a multi-platform RCP application with Tycho requires me to specify the platforms I want to build the application for. There's no way to do this in PDE so I specify the platforms so that the right fragments can be pulled into the RCP build. So we use as much as we can and augment where necessary. This is the general strategy that we've taken with the Maven integration. The latest versions of M2Eclipse delegate almost entirely to Eclipse when run inside the Eclipse. We basically stopped fighting Eclipse inside Eclipse because that was a losing battle. But there are times when we have to extend what's is normally done in Eclipse because otherwise it's inconvenient for users. The RCP app building is just one example.

> In case of Eclipse, you have project specific settings where you can provide fine-grained control over things like Compiler Warnings, Java compliance, Execution Environments, Code formatting, Clean up actions on save, and much, much more. AFAIK, no build tool provides an IDE independent way to define that. The same is true for Eclipse natures, internal builders, the containers of the .classpath, and build.properties (which enable bundle export using the build in PDE tooling).

There is always going to be some specific integration for particular IDEs. But with M2Eclipse configurators we can control almost everything. Then we talk to people like Milos who works on the Netbeans integration and we try to have similar behaviors for the Maven integration there. So we'll never be exact but we get relatively close considering no UI can ever be generated and the internals of the three major IDEs are entirely different. And just because the IDEs were not designed to interoperate doesn't mean we're not going to try to make them interoperate as best we can.

> 
> I think that the build tool should understand what an IDE provides.

We can agree to disagree there. The build needs at a base level to operate without any IDE and it's specifically because all the IDEs are different that this is not a good idea in practice. The CI builds need to execute regardless of IDE flavor and when you have teams using a mixed set of tools which is fairly common how would you decide which IDEs metadata to use? I've seen too many product builds being reliant on a particular IDE and this makes automation and release management extremely hard.

A build tool should be independent at a core level with all developer representations and should strive fidelity in representation of that to the developer in the IDE. In a large number of our clients the build and release teams don't even use an IDE. In the largest of organizations there are usually central build teams or SCM councils and if you told these people the build was dependent on one particular IDE to drive the automated build you wouldn't be there very long. This is 10 years of experience looking at the nastiest of builds, not conjecture.

> A non-intrusive build tool integration for an IDE is very much about understanding the meta-data that the IDE already have and use that data to its full extent. Ideally, it should also build using the same mechanisms that the IDE does.
> 

Again, we will have to disagree because that just riles any serious build automation engineer. I believe that type of coupling is categorically wrong. I believe fidelity is required, I believe that the core logic should be shared, but it has to have started outside the IDE and then embedded in the IDE to provide the same characteristics. Going from what exists in the IDE that has grown out organically is a problem. This is why so many people have problems with automated builds in Eclipse because much of it is so coupled to Eclipse internals. I personally do not believe running a headless Eclipse instance is a sane thing to do for CI for example which I see a lot.

But in the case of Tycho we try to leverage P2, JDT and OSGi as much as possible to produce the same results and as time goes by. The benefit of our approach is that we're not coupled to Eclipse and if the fellows working OSGi integration in IDEA want to use Tycho as a tool which can be represented in IDEA and integrated then that's a win. Then we have two IDEs that will by definition produce the same headless result. Same goes for Netbeans. Both already have Maven integration so I think this is the pattern that will prove out to be the most useful over time.

> Regards,
> Thomas Hallgren
> 
> _______________________________________________
> egit-dev mailing list
> egit-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/egit-dev

Thanks,

Jason

----------------------------------------------------------
Jason van Zyl
Founder,  Apache Maven
http://twitter.com/jvanzyl
----------------------------------------------------------