Bug 486614 - Test local build of new Orbit build system
Summary: Test local build of new Orbit build system
Status: NEW
Alias: None
Product: Orbit
Classification: Tools
Component: releng (show other bugs)
Version: unspecified   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Project Inbox CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-01-26 21:11 EST by David Williams CLA
Modified: 2016-01-27 13:58 EST (History)
1 user (show)

See Also:


Attachments
screen shot of config showing example (33.32 KB, image/png)
2016-01-27 12:41 EST, David Williams CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description David Williams CLA 2016-01-26 21:11:38 EST
One requirement of our new Orbit build system is that others can build it on other machines, "locally" so to speak, and not just on "build.eclipse.org" (Hipp instances).  

This bug will cover my efforts to do that, and I will also ask questions (or make suggestions :) along the way even if not directly related to "local build" per se.
Comment 1 David Williams CLA 2016-01-26 21:23:02 EST
My first question is "where do the jars come from"? 

I see in the readme.md file that is says, "This project uses Maven for assembling of OSGi bundles based on artifacts in Maven Central or
any other accessible Maven repository." 

But, that won't be true for all our jar files -- that is, they will not all be "in maven" in any repository.

Plus, even if a package is in maven central -- or if we "had our own" -- I think it is better practice to get the package from the original source -- say its Apache project -- rather than taking a derived work. 

Plus, even if in "maven central", we need to "get" our packages from an Eclipse controlled resource. 

These are major issues for "the new build" being a realistic production replacement for what we have now. And I do not know the status or the steps needed to solve these?
Comment 2 David Williams CLA 2016-01-26 21:25:56 EST
Another question: Why does each step of the build publish to repo.eclipse.org? 

I think it would be better to "publish" only certain builds -- or, at least, wait until the end to make sure the whole build is successful (and, eventually, passes all tests).
Comment 3 David Williams CLA 2016-01-26 21:27:02 EST
Another question, why are the bundles and feature and repository built separately? I'm sure there is a good reason -- just asking.
Comment 4 Gunnar Wagenknecht CLA 2016-01-27 11:54:08 EST
(In reply to David Williams from comment #1)
> My first question is "where do the jars come from"? 

Currently Maven Central. But they should not come from there -> bug 484631. There is also bug 484633 (currently blocking bug 484631).(In reply to David Williams from comment #2)

> Another question: Why does each step of the build publish to
> repo.eclipse.org? 

Can you give an example? There should be no publish to repo.eclipse.org.

-> `mvn clean install -f releng/aggregationfeature/pom.xml`

Installs generated bundles into your local Maven repository.

-> `mvn clean package -f releng/repository/pom.xml`

Aggregates bundles from the local Maven repository into the p2 repository.

(In reply to David Williams from comment #3)
> Another question, why are the bundles and feature and repository built
> separately? I'm sure there is a good reason -- just asking.

It's a technical limitation in Tycho. When Tycho performs the resolution, none of the bundles have been generated yet. It fails to see them. That's why they are split into separate Maven Reactors. The first run generates the bundles and places them into the local Maven repository. The second run allows Tycho to resolve them from the local Maven repository.
Comment 5 David Williams CLA 2016-01-27 12:41:58 EST
Created attachment 259418 [details]
screen shot of config showing example

(In reply to Gunnar Wagenknecht from comment #4)
> 
> > Another question: Why does each step of the build publish to
> > repo.eclipse.org? 
> 
> Can you give an example? There should be no publish to repo.eclipse.org.
> 

I see it in the "advanced settings" of each build step, as in attached screen capture. 

But, just looked in "repo.eclipse.org" and could not find anything published there under 'orbit', so maybe the "setting" is deceptive?
Comment 6 David Williams CLA 2016-01-27 12:46:40 EST
> (In reply to David Williams from comment #3)
> > Another question, why are the bundles and feature and repository built
> > separately? I'm sure there is a good reason -- just asking.
> 
> It's a technical limitation in Tycho. When Tycho performs the resolution,
> none of the bundles have been generated yet. It fails to see them. That's
> why they are split into separate Maven Reactors. The first run generates the
> bundles and places them into the local Maven repository. The second run
> allows Tycho to resolve them from the local Maven repository.

Hmmm, that must be because the bundles come from a maven repo. Right? 

For example, in the Eclipse Platform build, we easily build bundles and features "at the same time".
Comment 7 David Williams CLA 2016-01-27 12:56:36 EST
(In reply to Gunnar Wagenknecht from comment #4)
> (In reply to David Williams from comment #1)
> > My first question is "where do the jars come from"? 
> 
> Currently Maven Central. But they should not come from there -> bug 484631.
> There is also bug 484633 (currently blocking bug 484631).(In reply to David
> Williams from comment #2)
> 

Ok. Might be good to have a working example of a "raw jar", even if it temporarily exists simply on the file system somewhere? Basically, I would feel better seeing everything work when the package was was not a "mavenized jar".
Comment 8 Gunnar Wagenknecht CLA 2016-01-27 13:58:17 EST
(In reply to David Williams from comment #5)
> Created attachment 259418 [details]

> I see it in the "advanced settings" of each build step, as in attached
> screen capture. 


That is just a configuration option in Hudson in case a project _wants_ to publish something to repo.eclipse.org. It gives Maven/Hudson the proper permissions to do so. Can be ignored. :)

(In reply to David Williams from comment #6)
> > (In reply to David Williams from comment #3)
> > > Another question, why are the bundles and feature and repository built
> > > separately? I'm sure there is a good reason -- just asking.
> > 
> > It's a technical limitation in Tycho. When Tycho performs the resolution,
> > none of the bundles have been generated yet. It fails to see them. That's
> > why they are split into separate Maven Reactors. The first run generates the
> > bundles and places them into the local Maven repository. The second run
> > allows Tycho to resolve them from the local Maven repository.
> 
> Hmmm, that must be because the bundles come from a maven repo. Right? 
> 
> For example, in the Eclipse Platform build, we easily build bundles and
> features "at the same time".

That's different. In the Eclipse Platform build you build the bundles with Tycho. Thus, Tycho knows from the very beginning of the build that these are bundles. EBR does not. EBR generates bundles as part of the build from Maven libraries. When Tycho is doing a resolution, that generation has not happened yet.

For background see bug 353889.

(In reply to David Williams from comment #7)
> (In reply to Gunnar Wagenknecht from comment #4)
> > (In reply to David Williams from comment #1)
> > > My first question is "where do the jars come from"? 
> > 
> > Currently Maven Central. But they should not come from there -> bug 484631.
> > There is also bug 484633 (currently blocking bug 484631).(In reply to David
> > Williams from comment #2)
> > 
> 
> Ok. Might be good to have a working example of a "raw jar", even if it
> temporarily exists simply on the file system somewhere? Basically, I would
> feel better seeing everything work when the package was was not a "mavenized
> jar".


In order to see it working with a "normal jar" we need bug 484631 (and possible agree on a process, bug 484633). The reason is that a normal jar would no longer be checked into Git but uploaded to repo.eclipse.org for consumption by Orbit (or any other project with a CQ).