Community
Participate
Working Groups
Created attachment 223396 [details] sample maven tyhco project that reproduces the problem Using tycho 0.16.0 when we reference a jar in the "Bundle-ClassPath:" in the manifest file, and the jar was copied by copied by the maven-dependency-plugin, the build will fail at least half the time. This failure does not happen when we use tycho 0.15.0, it also doesn't happen if the jar is not referenced in the manifest file. The attached maven project can be used to reproduce this bug. You may need to run "mvn clean install" a few time on the project before you manage to reproduce the bug. Here is sample output of the failure: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin: 2.1:copy-dependencies (copy-bundle-classpath-libs) on project com.ptc.appmanager .errorsample: Error copying artifact from C:\Users\gisraeli.PTCNET\Desktop\tycho \tycho\target\driver\jtds.jar to C:\Users\gisraeli.PTCNET\Desktop\tycho\tycho\ta rget\driver\com.ptc.appmanager.errorsample-target\driver\jtds.jar.jar: File C:\U sers\gisraeli.PTCNET\Desktop\tycho\tycho\target\driver\jtds.jar does not exist - > [Help 1] (note the part where it is trying to copy the artifact "jtds.jar.jar" instead of "jtds.jar")
bin.includes = META-INF/,\ .,\ target/driver/jtds.jar and Bundle-ClassPath: target/driver/jtds.jar, is at least questionable. It may have worked by chance before. Copy it e.g. to <projectBaseDir>/lib/jtds.jar and reference it as bin.includes=lib/jtds.jar Bundle-ClassPath: lib/jtds.jar
actually it used to be: bin.includes = META-INF/,\ .,\ driver/jtds.jar and Bundle-ClassPath: driver/jtds.jar, but we saw the same bug
(In reply to comment #2) > actually it used to be: > > bin.includes = META-INF/,\ > .,\ > driver/jtds.jar > > and > Bundle-ClassPath: driver/jtds.jar, > > but we saw the same bug please provide a sample projects and exact steps to reproduce the problem. "build will fail at least half the time" is not enough
In our production environment the build fails every time. I attached a zip to the bug containing a minimal sample project that reproduces the bug. In the sample project the first time you run "mvn clean install" the first time you build the project it will succeed, but the second time it will fail, the third time will succeed, and the forth will fail, etc... (it seems to follow this pattern in a fairly deterministic fashion on my machine at least...)
the attached file is called "sample maven tyhco project that reproduces the problem "
(In reply to comment #2) > actually it used to be: > > bin.includes = META-INF/,\ > .,\ > driver/jtds.jar > > and > Bundle-ClassPath: driver/jtds.jar, > > but we saw the same bug I told you the current version using target/ dir is questionable. Provide a sample of the version that "used to be:" together with proof that it does not work.
Created attachment 223421 [details] Updated sample maven tyhco project that reproduces the problem (no longer uses target folder)
sorry forgot to upload the file :P the attachment named "Updated sample maven tyhco project that reproduces the problem (no longer uses target folder)" should now contain the project that causes the problem without using the target/ folder here is the updated output: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.1:copy-dependencies (copy-bundle-classpath-libs) on project com.ptc.appmanager.errorsample: Error copying artifact from C:\Users\gisraeli.PTCNET\Desktop\tycho\tycho\driver\jtds.j ar to C:\Users\gisraeli.PTCNET\Desktop\tycho\tycho\driver\com.ptc.appmanager.errorsample-driver\jtds.jar.jar: File C:\Users\gisrae li.PTCNET\Desktop\tycho\tycho\driver\jtds.jar does not exist -> [Help 1]
Created attachment 223461 [details] fixed sample project
probably a bug in maven-dependency-plugin attached fixed version of your sample project works. every time.
cool thanks :)
Hello everybody, I am running into the same problem as Gilad when switching my build system from Tycho 0.14.1 to Tycho 0.16.0. The maven-dependency-plugin fails in the copy-dependency operation about every second build with the same type of error message that a file could not be copied. My configuration differs slightly from the fixed demo project, but even when using the one of the demo project, the error remained: <plugin> <!-- Plugin for copying jar files of dependencies to lib and lib-src folders. --> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-dependency-plugin</artifactId> <version>2.1</version> <executions> <execution> <!-- Execution for copying jar files of dependencies to lib folder. --> <id>copy-dependencies</id> <!-- Make sure this runs before compile! --> <phase>process-resources</phase> <goals> <goal>copy-dependencies</goal> </goals> <configuration> <outputDirectory>lib</outputDirectory> <!-- Exclude all artifacts originating from plugin dependencies since we do not want them copied. Eventually, we do need another filter pattern here if we include further plugin dependencies. --> <excludeGroupIds>p2.eclipse-plugin</excludeGroupIds> <excludeTransitive>true</excludeTransitive> </configuration> </execution> <execution> <!-- Execution for copying sources-jar files of dependencies to lib-src folder. --> <id>copy-dependencies-sources</id> <!-- Make sure this runs before compile! --> <phase>process-resources</phase> <goals> <goal>copy-dependencies</goal> </goals> <configuration> <classifier>sources</classifier> <outputDirectory>lib-src</outputDirectory> <!-- Exclude all artifacts originating from plugin dependencies since we do not want them copied. Eventually, we do need another filter pattern here if we include further plugin dependencies. --> <excludeGroupIds>p2.eclipse-plugin</excludeGroupIds> <!-- Do not fail if there is no sources jar for a particular dependency. --> <failOnMissingClassifierArtifact>false</failOnMissingClassifierArtifact> <excludeTransitive>true</excludeTransitive> </configuration> </execution> </executions> </plugin> Do you have any hints what might be wrong? Maybe we should reopen this bug report... Thanks for any help. Timo
(In reply to comment #12) > Hello everybody, > Do you have any hints what might be wrong? Maybe we should reopen this bug > report... upgrade to latest maven-dependency-plugin version 2.6 may help. If it doesn't work and you want to you reopen, make sure to provide a minimal sample project along with steps to reproduce/expected versus observed behaviour.
Unfortunately, updating to version 2.6 of the maven-dependency-plugin did not work - I already tried that before and now once again. What seems a little weired to me is the actual error message that I am getting with version 2.6 now: Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.6:copy-dependencies (copy-dependencies) on project com.initplan.mplan.server.legacy: Error copying artifact from C:\development\workspaces_mpng\mpng_server_rmi_trunk\com.initplan.mplan.server.legacy\lib\databasex-1.8.1-salt.jar to C:\development\workspaces_mpng\mpng_server_rmi_trunk\com.initplan.mplan.server.legacy\lib\com.initplan.mplan.server.legacy.source-1.0.0.qualifier-lib\databasex-1.8.1-salt.jar.jar: File C:\development\workspaces_mpng\mpng_server_rmi_trunk\com.initplan.mplan.server.legacy\lib\databasex-1.8.1-salt.jar does not exist -> [Help 1] Notice the location to where it wants to copy the JAR-file: This "com.initplan.mplan.server.legacy.source-1.0.0.qualifier-lib" subfolder does not even exist and also should not exist! However, it is true that the source file it wants to copy also does not exist whenever the error occurs. But what I don't understand is why it tries to copy a file from a location to the same location... Creating a demo project is a little difficult for me as you won't be able to satisfy the dependencies which are only available in our own JFrog Artifactory... Do you happen to have any further hint that I could investigate?
By the way, Jan, I tried again the sample project of Gilad that you fixed to get an idea what I could do next. Unfortunately, it does not work for me either... Every second build, it fails with a similar error message as with Gilad: Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:2.5.1:copy-dependencies (copy-bundle-classpath-libs) on project org.tycho.sample.fixed: Error copying artifact from C:\development\workspaces_mpng\tycho-copy-dependencies\org.tycho.sample.fixed\driver\jtds-1.2.4.jar to C:\development\workspaces_mpng\tycho-copy-dependencies\org.tycho.sample.fixed\driver\org.tycho.sample.fixed-1.0.0.qualifier-driver\jtds-1.2.4.jar.jar: File C:\development\workspaces_mpng\tycho-copy-dependencies\org.tycho.sample.fixed\driver\jtds-1.2.4.jar does not exist -> [Help 1]
Hello, I definitely don't know why, but with the fixed demo project of Gilad, it is as follows on my machine: - Starting with a clean project (mvn clean), i.e. no driver and no target subfolder. - Running "mvn clean install" works the first time. - Running "mvn clean install" fails the second time. - Running "mvn clean install" works the third time. - Running "mvn clean install" fails the fourth time. - etc. But everything works fine (again starting from a clean project) if I take away the references to driver/jtds-1.2.4.jar from the MANIFEST.MF - but that's not what I want, I do need the libraries referenced there... Any further hints? Thanks again!
I cannot reproduce the problem with the fixed demo project (with stripVersion=false), but the problem is reproducible with stripVersion=true (and any of maven-dependency-plugin 2.5.1/2.6 and Tycho 0.16.0/0.17.0-SNAPSHOT). The stack trace in case of the failure [1] however points out that Tycho does in fact have a fundamental problem: Tycho seems to add a dependency (in the Maven model) to the Bundle-ClassPath element if and only if the referenced jar file exists. This is the case after the first successful build, and not the case after a mvn clean (or a failed build). I believe the "exists" check means to cover the cases where the referenced jar is or is not part of the sources. (Bundle-ClassPath entries do not need to exist, they may be contributed at runtime through fragments.) The logic obviously fails if the jar is only created during the build. There may be other solutions, but the problem will also be fixed with bug 353889: Once the dependency resolution & class path computation happens as part of the normal build, the clean plug-in and dependency-plugin will have been executed before Tycho checks the jar existence. [1] Stack trace: org.apache.maven.plugin.MojoExecutionException: Error copying artifact from C:\Env\Test\tycho-bugs\393978-maven-dependency-interaction\tycho\driver\jtds.jar to C:\Env\Test\tycho-bugs\393978-maven-dependency-interaction\tycho\driver\com.ptc.appmanager.errorsample-driver\jtds.jar.jar at org.apache.maven.plugin.dependency.AbstractDependencyMojo.copyFile(AbstractDependencyMojo.java:196) at org.apache.maven.plugin.dependency.CopyDependenciesMojo.copyArtifact(CopyDependenciesMojo.java:223) at org.apache.maven.plugin.dependency.CopyDependenciesMojo.execute(CopyDependenciesMojo.java:116) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) ... 19 more Caused by: java.io.IOException: File C:\Env\Test\tycho-bugs\393978-maven-dependency-interaction\tycho\driver\jtds.jar does not exist at org.codehaus.plexus.util.FileUtils.copyFile(FileUtils.java:1039) at org.apache.maven.plugin.dependency.AbstractDependencyMojo.copyFile(AbstractDependencyMojo.java:192) ... 23 more
*** Bug 403718 has been marked as a duplicate of this bug. ***
(In reply to comment #18) > *** Bug 403718 has been marked as a duplicate of this bug. *** Contains another sample project (where the build doesn't fail).
Just to clarify is this a tycho issue or a maven-dependency-plugin issue?
(In reply to comment #20) > Just to clarify is this a tycho issue or a maven-dependency-plugin issue? See comment #17
Has there been any progress here? Is there currently ANY way to use tycho with non-OSGi libraries that are NOT checked out from version control, but downloaded from an artifact repository? If there isn't we simply can not use tycho... This is a blocker to us unless there is an acceptable workaround. Is there any way we can help to speed things up?
Please consider setting importance to critical unless there is a workaround available - in which case I would like to know this workaround ;-)
I use maven-bundle-plugin to embed maven dependencies in OSGi bundles, then use http://wiki.eclipse.org/Tycho/How_Tos/Dependency_on_pom-first_artifacts to reference these bundles from my Tycho builds. You can see m2e build setup for an example. Works well enough to justify major (!) development effort necessary to embed maven dependencies in Tycho builds directly.
(In reply to Jörg Sesterhenn from comment #22) > Has there been any progress here? here is my understanding of what is going wrong (you may use the "fixed" sample project to verify it) 1. Tycho injects system-scoped dependencies into the maven model to make bundle jars (and nested jars) available to the maven classpath model. This is only for interop with other maven plugins which are not aware of tycho/OSGi so they can reuse the compile classpath. As an added complication, system-scoped dependencies are added at the very start of the build and *only if they exist* at the very start of the build. If you copy binaries during build, they may not exist yet. This is why you get the alternating behaviour mentioned above. This issue will only be solved with bug 353889 2. the copy-dependencies goal of m-d-p [1] will copy all effective dependencies of the module it runs in (including the system-scoped ones injected by tycho). This is where we get into some kind of cycle that causes the strange ".jar.jar" names. However, in this scenario with OSGi wrappers using nested jars, I suppose that what you really want is to copy a dedicated (list of) maven artifact(s), that's it. I suggest to _avoid_ using the copy-dependencies goal for reasons given above and use the alternative "copy" goal [2] instead which takes an explicit list of GAVs and does not use the project's dependencies. This is what we do in tycho itself, see [3] for example and we did not face any issues up to now. [1] http://maven.apache.org/plugins/maven-dependency-plugin/copy-dependencies-mojo.html [2] http://maven.apache.org/plugins/maven-dependency-plugin/copy-mojo.html [3] http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-surefire/org.eclipse.tycho.surefire.junit/pom.xml
(In reply to Jan Sievers from comment #25) > http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-surefire/org.eclipse.tycho.surefire.junit/pom.xml to understand the example, you will also need the configuration inherited from parent pom http://git.eclipse.org/c/tycho/org.eclipse.tycho.git/tree/tycho-surefire/pom.xml
The workaround of using copy instead of copy-dependencies works for a command-line build. Would be nice to be able to copy transitive dependencies as well, though (AFAIK copy goal doesn't support them). The workaround does NOT work for M2E -- e.g. try to repeatedly Maven -> Update Project and it will still fail every second time.
(In reply to Erkki Lindpere from comment #27) > The workaround does NOT work for M2E -- e.g. try to repeatedly Maven -> > Update Project and it will still fail every second time. did you try m2e fix http://dev.eclipse.org/mhonarc/lists/m2e-users/msg04322.html ?
(In reply to Jan Sievers from comment #25) > (In reply to Jörg Sesterhenn from comment #22) > > Has there been any progress here? > > here is my understanding of what is going wrong (you may use the "fixed" > sample project to verify it) > > 1. Tycho injects system-scoped dependencies into the maven model to > make bundle jars (and nested jars) available to the maven classpath model. > This is only for interop with other maven plugins which are not aware of > tycho/OSGi so they can reuse the compile classpath. > As an added complication, system-scoped dependencies are added at the > very > start of the build and *only if they exist* at the very start of the > build. > If you copy binaries during build, they may > not exist yet. This is why you get the alternating behaviour mentioned > above. This issue will only be solved with bug 353889 > 2. the copy-dependencies goal of m-d-p [1] will copy all effective > dependencies > of the module it runs in (including the system-scoped ones injected by > tycho). > This is where we get into some kind of cycle that causes the strange > ".jar.jar" names. Do you think it would be sufficient to just not include system-scoped dependencies while using copy-dependencies (e.g. by specifying includeScope=runtime)? That way we would not run into this cycle... Or are there further differences to the copy goal? > However, in this scenario with OSGi wrappers using nested jars, I suppose > that what you really want is to copy a dedicated (list of) maven > artifact(s), that's it. > > I suggest to _avoid_ using the copy-dependencies goal for reasons given > above and use the alternative "copy" goal [2] instead which takes an > explicit list of GAVs and does not use the project's dependencies. This is > what we do in tycho itself, see [3] for example and we did not face any > issues up to now. Using copy is not really an option to me - I also need transitive runtime dependencies and do not want to list those (as this would be redundant and as such error prone).
(In reply to Jörg Sesterhenn from comment #29) > Do you think it would be sufficient to just not include system-scoped > dependencies while using copy-dependencies (e.g. by specifying > includeScope=runtime)? did not test this but <excludeScope>system</excludeScope> may also work around the problem.
(In reply to comment #30) > did not test this but > > <excludeScope>system</excludeScope> > > may also work around the problem. I've heard that this solves the problem.
(In reply to comment #30) > (In reply to Jörg Sesterhenn from comment #29) > > Do you think it would be sufficient to just not include system-scoped > > dependencies while using copy-dependencies (e.g. by specifying > > includeScope=runtime)? > > did not test this but > > <excludeScope>system</excludeScope> > > may also work around the problem. I've just tried this out, and adding this configuration doesn't make a difference for me. [1] Instead, I noticed that this bug occur if and only if copy-dependencies writes into a location that is deleted by the clean goal. So if you configure <outputDirectory>lib</outputDirectory> the copy-dependencies approach works, and if you configure <outputDirectory>target/lib</outputDirectory>, this bug occurs. So this is one workaround for this problem. And as said before, another potential workaround is to use the copy goal instead of copy-dependencies [2]. [1] https://github.com/oberlies/tycho-testbed/commit/721ae11180173d2c70799e252e13f723c61150c8 [2] https://github.com/oberlies/tycho-testbed/compare/copyLibrary_393978
It is a maven bug. From my point of view. Workaround: Call maven only with one goal. >mvn clean >mvn install instead >mvn clean install
(In reply to Jan Sievers from comment #6) > I told you the current version using target/ dir is questionable. Hi, we have the same issue and I would like to reply that using target is maven-compliant and follow its best-practices. We don't want lib to be added in Git (we have target excluded in .gitignore, althought I could add lib too). Also, having maven handled the lib directory with clean goal is good.
Eclipse Tycho is moving away from this bugs.eclipse.org issue tracker to https://github.com/eclipse/tycho/issues/ instead. If this issue is relevant to you, your action is required. 0. Verify this issue is still happening with latest Tycho 2.4.0-SNAPSHOT if issue has disappeared, please change status of this issue to "CLOSED WORKFORME" with some details about your testing environment and how you did verify the issue; and you're done if issue is still present when latest release: * Create a new issue at https://github.com/eclipse/tycho/issues/ ** Use as title in GitHub the title of this Bugzilla ticket (may include the bug number or not, at your own convenience) ** In the GitHub description, start with a link to this bugzilla ticket ** Optionally add new content to the description if it can helps towards resolution ** Submit GitHub issue * Update bugzilla ticket ** Add to "See also" property (up right column) the link to the newly created GitHub issue ** Add a comment "Migrated to <link-to-newly-created-GitHub-issue>" ** Set status as CLOSED MOVED ** Submit All issues that remain open will be automatically closed next week or so. Then the Bugzilla component for Tycho will be archived and made read-only.
