Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [dash-dev] First conversion of 3.6.2 projects

Am 18.03.2011 23:56, schrieb Alex Blewitt:

>> You can find the result in
>> /home/nexus/workspace/org.eclipse.dash.m4e.tools/tmp/m2repo/
>
> Looks good from what I've seen briefly. A couple of thoughts:
>
>     <dependency>
>       <groupId>org.eclipse.jface</groupId>
>       <artifactId>org.eclipse.jface</artifactId>
>       <version>[3.2.0,4.0.0)</version>
>       <optional>false</optional>
>     </dependency>
>
> We can probably optimise out the <optional>false</optional> if it's
optional.

Yes.

> Secondly, how does Maven handle the version range dependency? I know
technically it should support it, but I am concerned it might not.

Maven creates local metadata files which contain the available versions
on a server and then tries to find a match. Unlike p2, this happens
instantly, so it's not a performance issue.

> We *should* be able to find out what the release version is based on
> what we know is in the repository. For example,
> org/eclipse/jface/org.eclipse.jface/3.6.2 exists, so we could replace
> [3.2.0,4.0.0) with 3.6.2. This might make it easier for direct
> resolutions, as well as recording what was actually compiled
> against.

I think there is a bit more to that. If you see a version range, then
that version range was in the manifest. Without a version range, the
conversion step selects what it can find in the plugins/ directory that
is being converted. Examples:

org.jdom:org.jdom:1.0.0 wants org.apache.xerces but no version was
specified and this dependency is not available and it's optional. Result:

    <dependency>
      <groupId>org.apache.xerces</groupId>
      <artifactId>org.apache.xerces</artifactId>
      <version>[0,)</version>
      <optional>true</optional>
    </dependency>

org.eclipse.pde.build-3.6.2.pom contains ranges and fixed versions. For
example the dependency for org.eclipse.equinox.p2.publisher has no
version range in the MANIFEST.MF but it is available during the
conversion, so it's fixed to 1.1.0:

    <dependency>
      <groupId>org.eclipse.equinox</groupId>
      <artifactId>org.eclipse.equinox.p2.publisher</artifactId>
      <version>1.1.0</version>
      <optional>true</optional>
    </dependency>

There are several goals here:

- Preserve the original version range so people can see the minimum
dependency which still satisfies the needs of an artifact
- Stable builds
- Automatic updates

I see several solutions:

- Use properties for versions so consumers can override them
- Omit all versions and create one or more parent POMs with huge
dependencyManagement elements to specify the default versions
- Create profiles with the dependency versions. One profile uses the
range, the other uses the version which is in the same conversion.

Open issues:

- How do we resolve conflicts?
- How do we merge updates?

Resolving conflicts:

Imagine there is a.b.c:3.6.0 and e.f.g:3.6.0.

a.b.c:3.6.0 wants x.y.z:3.5.0 while e.f.g:3.6.0 wants x.y.z:3.*6*.0.

Example: org.apache.commons.logging comes as 1.0.4 and 1.1.1. For
org.w3c.dom.smil, we have 1.0.0 and 1.0.1. javax.xml.bind comes as 2.0.0
and 2.2.0.

Should we:

- Notify the team behind a.b.c to update their dependencies? What if
they refuse?
- Update the dependency ourselves (probably without knowing what we're
breaking)?
- Keep two versions of x.y.z?

Merging updates:

Say we have imported 3.6.0 into Nexus. Now, SR1 is released which
includes x.y.z_3.6.1. a.b.c:3.6.0 needs it but didn't specify a version
range. There is no a.b.c:3.6.1.

Should we

- release a.b.c:3.6.0.1 and make it depend on x.y.z_3.6.1?
- Update the parent POM and release everything in 3.6.0 as 3.6.0.1?
- Release a a.b.c:3.6.*1* even though there wasn't one?
- Create a report which lists such cases to consumers can make an
educated decision themselves?
- Leave it to the consumers to figure out?
-

>> Due to a bug, the repo is still polluted with non-Eclipse artifacts.
>
> No problem - we can probably just miss out anything that isn't in the
org/eclipse tree.

I cleaned the repo but there is still some artifacts outside of org/eclipse:

./com/ibm/icu/com.ibm.icu/4.2.1/com.ibm.icu-4.2.1.pom
./com/jcraft/jsch/com.jcraft.jsch/0.1.41/com.jcraft.jsch-0.1.41.pom
./com/lowagie/text/com.lowagie.text/2.1.7/com.lowagie.text-2.1.7.pom
./javax/xml/stream/javax.xml.stream/1.0.1/javax.xml.stream-1.0.1.pom
./javax/xml/bind/javax.xml.bind/2.0.0/javax.xml.bind-2.0.0.pom
./javax/xml/bind/javax.xml.bind/2.2.0/javax.xml.bind-2.2.0.pom
./javax/xml/soap/javax.xml.soap/1.2.0/javax.xml.soap-1.2.0.pom
./javax/xml/javax.xml/1.3.4/javax.xml-1.3.4.pom
./javax/xml/rpc/javax.xml.rpc/1.1.0/javax.xml.rpc-1.1.0.pom
./javax/transaction/javax.transaction/1.1.1/javax.transaction-1.1.1.pom
./javax/servlet/source/javax.servlet.source/2.5.0/javax.servlet.source-2.5.0.pom
./javax/servlet/javax.servlet/2.5.0/javax.servlet-2.5.0.pom
./javax/servlet/jsp/javax.servlet.jsp/2.0.0/javax.servlet.jsp-2.0.0.pom
./javax/persistence/javax.persistence/2.0.1/javax.persistence-2.0.1.pom
./javax/mail/javax.mail/1.4.0/javax.mail-1.4.0.pom
./javax/wsdl/javax.wsdl/1.5.1/javax.wsdl-1.5.1.pom
./javax/activation/javax.activation/1.1.0/javax.activation-1.1.0.pom
./java_cup/runtime/java_cup.runtime/0.10.0/java_cup.runtime-0.10.0.pom
./org/objectweb/asm/org.objectweb.asm/3.2.0/org.objectweb.asm-3.2.0.pom
./org/apache/jasper/org.apache.jasper/5.5.17/org.apache.jasper-5.5.17.pom
./org/apache/ws/org.apache.ws.commons.util/1.0.0/org.apache.ws.commons.util-1.0.0.pom
./org/apache/batik/org.apache.batik.svggen/1.6.0/org.apache.batik.svggen-1.6.0.pom
./org/apache/batik/org.apache.batik.transcoder/1.6.0/org.apache.batik.transcoder-1.6.0.pom
./org/apache/batik/org.apache.batik.dom.svg/1.6.0/org.apache.batik.dom.svg-1.6.0.pom
./org/apache/batik/org.apache.batik.xml/1.6.0/org.apache.batik.xml-1.6.0.pom
./org/apache/batik/org.apache.batik.util.gui/1.6.0/org.apache.batik.util.gui-1.6.0.pom
./org/apache/batik/org.apache.batik.util/1.6.0/org.apache.batik.util-1.6.0.pom
./org/apache/batik/org.apache.batik.pdf/1.6.0/org.apache.batik.pdf-1.6.0.pom
./org/apache/batik/org.apache.batik.parser/1.6.0/org.apache.batik.parser-1.6.0.pom
./org/apache/batik/org.apache.batik.dom/1.6.0/org.apache.batik.dom-1.6.0.pom
./org/apache/batik/org.apache.batik.bridge/1.6.0/org.apache.batik.bridge-1.6.0.pom
./org/apache/batik/org.apache.batik.css/1.6.0/org.apache.batik.css-1.6.0.pom
./org/apache/batik/org.apache.batik.ext.awt/1.6.0/org.apache.batik.ext.awt-1.6.0.pom
./org/apache/xml/org.apache.xml.serializer/2.7.1/org.apache.xml.serializer-2.7.1.pom
./org/apache/xml/org.apache.xml.resolver/1.2.0/org.apache.xml.resolver-1.2.0.pom
./org/apache/lucene/org.apache.lucene/1.9.1/org.apache.lucene-1.9.1.pom
./org/apache/lucene/org.apache.lucene.analysis/1.9.1/org.apache.lucene.analysis-1.9.1.pom
./org/apache/axis/org.apache.axis/1.4.0/org.apache.axis-1.4.0.pom
./org/apache/wsil4j/org.apache.wsil4j/1.0.0/org.apache.wsil4j-1.0.0.pom
./org/apache/oro/org.apache.oro/2.0.8/org.apache.oro-2.0.8.pom
./org/apache/bcel/org.apache.bcel/5.2.0/org.apache.bcel-5.2.0.pom
./org/apache/commons/org.apache.commons.discovery/0.2.0/org.apache.commons.discovery-0.2.0.pom
./org/apache/commons/org.apache.commons.logging/1.0.4/org.apache.commons.logging-1.0.4.pom
./org/apache/commons/org.apache.commons.logging/1.1.1/org.apache.commons.logging-1.1.1.pom
./org/apache/commons/org.apache.commons.lang/2.3.0/org.apache.commons.lang-2.3.0.pom
./org/apache/commons/org.apache.commons.httpclient/3.1.0/org.apache.commons.httpclient-3.1.0.pom
./org/apache/commons/org.apache.commons.net/2.0.0/org.apache.commons.net-2.0.0.pom
./org/apache/commons/org.apache.commons.el/1.0.0/org.apache.commons.el-1.0.0.pom
./org/apache/commons/org.apache.commons.codec/1.3.0/org.apache.commons.codec-1.3.0.pom
./org/apache/commons/org.apache.commons.collections/3.2.0/org.apache.commons.collections-3.2.0.pom
./org/apache/log4j/org.apache.log4j/1.2.15/org.apache.log4j-1.2.15.pom
./org/apache/xalan/org.apache.xalan/2.7.1/org.apache.xalan-2.7.1.pom
./org/apache/derby/org.apache.derby/10.5.1/org.apache.derby-10.5.1.pom
./org/apache/derby/org.apache.derby.core/10.5.1/org.apache.derby.core-10.5.1.pom
./org/apache/xerces/org.apache.xerces/2.9.0/org.apache.xerces-2.9.0.pom
./org/apache/velocity/org.apache.velocity/1.5.0/org.apache.velocity-1.5.0.pom
./org/apache/xmlrpc/org.apache.xmlrpc/3.0.0/org.apache.xmlrpc-3.0.0.pom
./org/apache/ant/org.apache.ant/1.7.1/org.apache.ant-1.7.1.pom
./org/uddi4j/org.uddi4j/2.0.5/org.uddi4j-2.0.5.pom
./org/mozilla/javascript/org.mozilla.javascript/1.7.2/org.mozilla.javascript-1.7.2.pom
./org/hamcrest/core/org.hamcrest.core/1.1.0/org.hamcrest.core-1.1.0.pom
./org/w3c/dom/org.w3c.dom.smil/1.0.0/org.w3c.dom.smil-1.0.0.pom
./org/w3c/dom/org.w3c.dom.smil/1.0.1/org.w3c.dom.smil-1.0.1.pom
./org/w3c/dom/org.w3c.dom.events/3.0.0/org.w3c.dom.events-3.0.0.pom
./org/w3c/dom/org.w3c.dom.svg/1.1.0/org.w3c.dom.svg-1.1.0.pom
./org/w3c/sac/org.w3c.sac/1.3.0/org.w3c.sac-1.3.0.pom
./org/w3c/css/org.w3c.css.sac/1.3.0/org.w3c.css.sac-1.3.0.pom
./org/junit/source/org.junit.source/4.8.1/org.junit.source-4.8.1.pom
./org/junit/org.junit/3.8.2/org.junit-3.8.2.pom
./org/sat4j/pb/org.sat4j.pb/2.2.0/org.sat4j.pb-2.2.0.pom
./org/sat4j/core/org.sat4j.core/2.2.0/org.sat4j.core-2.2.0.pom
./org/mortbay/jetty/org.mortbay.jetty.util/6.1.23/org.mortbay.jetty.util-6.1.23.pom
./org/mortbay/jetty/org.mortbay.jetty.server/6.1.23/org.mortbay.jetty.server-6.1.23.pom
./org/junit4/org.junit4/4.8.1/org.junit4-4.8.1.pom
./org/jdom/org.jdom/1.0.0/org.jdom-1.0.0.pom
./org/h2/org.h2/1.1.117/org.h2-1.1.117.pom
./net/sourceforge/lpg/net.sourceforge.lpg.lpgjavaruntime/1.1.0/net.sourceforge.lpg.lpgjavaruntime-1.1.0.pom
./lpg/runtime/java/lpg.runtime.java/2.0.17/lpg.runtime.java-2.0.17.pom
./commonj/sdo/commonj.sdo/2.1.1/commonj.sdo-2.1.1.pom

>> In the file, it looks like this:
>>
>>     948  Defl:N      517  46%  03-18-11 17:09  7ce61aa8
META-INF/MANIFEST.MF
>>     948  Defl:N      517  46%  03-18-11 17:15  7ce61aa8
META-INF/MANIFEST.MF
>>
>> So you can see the file date is different. How can that happen? I
>> understand that Eclipse recreates the MANIFEST.MF when the JAR is signed
>> but why are many plug-ins signed several times?
>
> There may be a bug in which a signed jar is already re-signed; this
wouldn't change the bits but would update the time processed.

I agree; but why are they signed again? Isn't there a cache or something?

>> Background: To make sure everything is ok, I check that all files are
>> identical when I merge repositories. Only, they aren't...
>
> Is the junit a special case? If so, we can probably ignore it.

No, I found 40 cases:

org.eclipse.jdt.debug-3.6.1.jar
org.eclipse.ui.intro.universal-3.2.402.jar
org.eclipse.core.runtime.compatibility.registry-3.3.0.jar
...

Regards,

-- 
Aaron "Optimizer" Digulla a.k.a. Philmann Dark
"It's not the universe that's limited, it's our imagination.
Follow me and I'll show you something beyond the limits."
http://blog.pdark.de/



Back to the top