Community
Participate
Working Groups
Currently if we have packaging 'eclipse-target-definition' tycho only installs the target platform into maven repository, this has some drawbacks: - if the target platform is used multiple times (across projects), it is resolved multiple times against remote servers - if an of the servers goes offline, the target can't be used - if any artifacts change on the remote sever, I get different results (non reproducible builds) - if I don't wan't net-io on each build I have to setup custom caching-servers to solve this, there should be a configuration option that enables the following: - on compile-phase the target is mirrored (the mirror-tool could be possibly used for this) into the builds output directory - on install-phase, all these artifacts are installed beside the target platform - on resolve, tycho should use the cached artifacts (if given) instead of those fetched from the remote pages If we are in install phase and there are already cached artifacts, tycho should only download artifacts if the have changed, and warn if the artifacts can't be refreshed due to server not reachable. Some of this is already possible but requires a lot of manual work/configuration: - You can create a mirror-goal that replicates all your entries in the target and mirror each update-site in the target to the build-dir, this gets really cumbersome when there are dependencies between repositories. - depyloing the result to a nexus raw-repository and define mirrors for the target sites that points to the nexus-location for reproducable builds - But must make sure that you don't use that mirror settings when building the target itself, so different settings are needed - must be repeated on each change of update-site
The use-case is pretty similar to what we had on JBoss Tools when I was working on this. In https://www.slideshare.net/mickaelistria/a-journey-with-target-platforms I explained the various tricks we did set up for more performant and more reproducible consumption of target-platforms, and they're more or less what you're asking for. Back then I wasn't so involved in Tycho, but now I am and I better see how Tycho could support those use-cases more naturally. Some ideas to consider: * Many target definitions are intentionally volatile and mutable, referencing repos with changing content. That's probably the main use-case and should remain the 1st class citizen (ie this should work without any change) * A target-platform resolution/mirror is thus a snapshot which is only correct at a given point of time, without guarantee it's still correct later (and this is true for any target-platform, independently of the content as we don't know whether a p2 repo is mutable or not), so it should be clear that the mirror and other artifacts related to the target platform are a snapshot (ie they could contain a resolution date) * The possible artifacts (mirror p2 site, derived .target...) should be attached to the same artifact as the eclipse-target-definition, but using different classifiers * In the end, it'd be good if the final artifact to consume would also be a .target file, so it means downstream, no adaptation would be necessary to consume it. About caching, Tycho already has a lot of cache optimized, so it wouldn't fetch artifacts that are already cached. Maybe we should somewhere document how this cache can be populated manually from an existing p2 repo. This could actually be a separate and independent feature which can help in other cases, so maybe better to track it in a separate ticket.
(In reply to Mickael Istria from comment #1) > Some ideas to consider: Thanks for sharing that I think you are right and my ideas are very simliar, and yes I'd like to support this in a "natural" way in tycho > Many target definitions are intentionally volatile and mutable, > referencing repos with changing content. That's probably the main use-case > and should remain the 1st class citizen (ie this should work without any > change) Even though I would consider that bad style (you should not develop against a moving target) I know its often the case so I have a configuration-option in mind to explicitly switch this feature on. > A target-platform resolution/mirror is thus a snapshot which is only > correct at a given point of time, without guarantee it's still correct later > (and this is true for any target-platform, independently of the content as > we don't know whether a p2 repo is mutable or not), so it should be clear > that the mirror and other artifacts related to the target platform are a > snapshot (ie they could contain a resolution date) What I have especially in mind (beside the possibility to deploy the result into e.g nexus) is to get closer to how PDE-Tooling is working (and why tycho is slow compared to using IDE): You once load the target and then "set as active target", at this point of time PDE stores the current state of the target and do not resolve on again on every build-request. a 'mvn install' of my target should work the same if the mirror feature is turned on. if I now use this target in builds (e.g. only building a sub-module of a project) tycho should use that resolved state and could be super-fast. We have a rather large build (> 100 bundles) and resolving the target (what is also rather large) uses about 90% of the buildtime. > The possible artifacts (mirror p2 site, derived .target...) should be > attached to the same artifact as the eclipse-target-definition, but using > different classifiers I have played around already a little bit with this and my idea is the following: - read in the target - create a mirror of all IUs in a subfolder - generate a composite repository descriptor in the root of the mirror folder referencing the subfolder - in case of deploy create a zip out of this and deply it with classifier "p2" or something like that. If tycho found such a p2/mirrored target it could simply ignore the target-file at all and use the folder/zip as source for the target as if it was given as a extra repository with layout p2, that simplifies things alot. > In the end, it'd be good if the final artifact to consume would also be a > .target file, so it means downstream, no adaptation would be necessary to > consume it. That all depends on how flexible P2/Tycho is in terms of "reading whole target from arbitrary source", even though it should be possible to extend support for this, currently P2/Tycho is very "file-system-centric" so I would suspect to at least add a little support downstream, but thats not so much to change as mentioned above the p2-repository-layout code can be used for that.
To support the "deploy a modified target" approach IMO we must have support for having mvn artifacts referenced in target plaftorms, I'm not sure how "open" the PDE/m2e guys are about extensions in that area, but I once created a showcase how maven artifacts (with or without OSGi-Metadata) could be used inside a PDE Targetplatform: https://github.com/laeubi/eclipseplugins/tree/master/target/de.laeubisoft.eclipseplugins.target.maven That would make it possible to add mvn dependecies to regular targets and thus remove the need for "considerMavenDependecies" as well as adding first-class support to developers using the target in an IDE.
Eclipse Tycho is moving away from this bugs.eclipse.org issue tracker to https://github.com/eclipse/tycho/issues/ instead. If this issue is relevant to you, your action is required. 0. Verify this issue is still happening with latest Tycho 2.4.0-SNAPSHOT if issue has disappeared, please change status of this issue to "CLOSED WORKFORME" with some details about your testing environment and how you did verify the issue; and you're done if issue is still present when latest release: * Create a new issue at https://github.com/eclipse/tycho/issues/ ** Use as title in GitHub the title of this Bugzilla ticket (may include the bug number or not, at your own convenience) ** In the GitHub description, start with a link to this bugzilla ticket ** Optionally add new content to the description if it can helps towards resolution ** Submit GitHub issue * Update bugzilla ticket ** Add to "See also" property (up right column) the link to the newly created GitHub issue ** Add a comment "Migrated to <link-to-newly-created-GitHub-issue>" ** Set status as CLOSED MOVED ** Submit All issues that remain open will be automatically closed next week or so. Then the Bugzilla component for Tycho will be archived and made read-only.
