Hi Christian,
How
did you monitor the time spent by maven ? Do
you have a report for each step in the
lifecycle?
One
solution to help maven would be refactor the
top-pom-main to have real module management.
For
example replace :
<modules>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.common.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.common.sdk.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.common.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.custom.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.custom.metamodel</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.custom.metamodel.edit</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.custom.sdk.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.custom.sdk.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.custom.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.catalog</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.edit</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.editor</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.metamodel</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.metamodel.edit</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.sdk.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.efacet.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.java</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.java.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.java.metamodel</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.java.sdk.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.ocl.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.ocl.metamodel</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.query.ocl.sdk.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.emf.catalog</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.emf.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.emf.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.jface.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.pde.core</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.swt</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.util.ui</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.widgets</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.widgets.celleditors</module>
<module>../plugins/facet/org.eclipse.papyrus.emf.facet.widgets.celleditors.ecore</module
</modules>
By
<modules>
<module>../plugins/facet</module>
</modules>
And
a pom.xml in the directory facet containing
all modules
A
degraded solution would be to help maven by
listing module in the order they will be
build.
=>Ex
: moving the features at the end of the
modules list
Regards,
Benoit
Hi,
A
single POM would probably help in this case.
Invalid builds would probably fail later
(Because dependencies from the 4 builds would
have to be resolved before trying to compile
anything), but valid builds would probably
take less time, so that would help.
Camille
De :
mdt-papyrus.dev-bounces@xxxxxxxxxxx
[mailto:mdt-papyrus.dev-bounces@xxxxxxxxxxx]
De la part de Christian W. Damus
Envoyé : vendredi 13 février 2015 13:31
À : Papyrus Project list
Objet : Re: [mdt-papyrus.dev] About
tests
I,
too, have observed how much time is spent in
dependencies. I wonder, is it a maven problem
or a Tycho problem? It's an order of
magnitude more than the compile step. In any
case, do you think any significant time could
be saved by running a single maven build on a
pom that is parent to all four of these
builds?
On Fri, Feb 13, 2015 at 7:20 AM, LETAVERNIER
Camille <Camille.LETAVERNIER@xxxxxx>
wrote:
Hi,
The
Gerrit job is back to normal. The
failures due to the invalid Job
configuration (should) have been
retriggered.
Unfortunately,
even just compiling tests takes an
additional 15-20 minutes (Most of the
time is spent in the dependencies
resolution rather than actually
compiling). So Gerrit jobs now take a
total of 30-40 minutes depending on
the server load, and the queue just
keeps growing. The total time between
a Gerrit contribution and the Hudson
validation may be close to 1 hour now
(Thats still much lower than the time
between the contribution and the
actual committer review however :) )
Hi
all,
Bug
459800 [1] has been reported to
compile main/tests + extra/tests
during the Gerrit build. Im
currently working on it and started
a test build. If I missed anything,
other Gerrit builds in progress may
receive a -1 from Hudson. If this
happens, this may mean that the b
job configuration is invalid, rather
than your contribution.
Ill
send and update when Im sure the
Gerrit job is properly configured to
compile all tests. Also note that
test execution is not yet planed,
for performances reasons (Running
all tests takes more than an hour,
while current Gerrit builds take
less than 20 minutes)
[1]
459800: [Releng] Gerrit review
builds should compile all tests
https://bugs.eclipse.org/bugs/show_bug.cgi?id=459800
Regards,
Camille
Gerrit
compiles Main + Extra, and skips the
tests
Camille
De :
mdt-papyrus.dev-bounces@xxxxxxxxxxx
[mailto:mdt-papyrus.dev-bounces@xxxxxxxxxxx]
De la part de Christian W.
Damus
Envoyé : jeudi 12 février 2015
16:35
À : Papyrus Project list
Objet : Re: [mdt-papyrus.dev]
About tests
Reversion
is pushed and a new tests build is
scheduled.
It
seems that our Gerrit review builds
not only dont run the tests, but they
also dont compile them. Do they at
least compile the Extras plug-ins?
What if a Gerrit patch includes
changes in Extras plug-ins?
On Thu, Feb 12, 2015 at 10:22 AM,
LETAVERNIER Camille <Camille.LETAVERNIER@xxxxxx>
wrote:
Hi
all,
Quick update, since things
have changed quite a bit just
after I posted the initial
report:
-
311
new failures in the test
results:
o
The
issue is known and being
worked on (The SysML BDD
Package is broken, causing all
the failures)
-
Test
build dont compile anymore
o
Is
anyone working on that?
-
Failed to execute
goal
org.eclipse.tycho:tycho-compiler-plugin:0.22.0:compile
(default-compile) on project
org.eclipse.papyrus.infra.core.sasheditor.tests:
Compilation failure:
Compilation failure:
-
[ERROR]
/jobs/genie.modeling.mdt.papyrus/Papyrus-Master-Tests/workspace/tests/junit/plugins/core/org.eclipse.papyrus.infra.core.sasheditor.tests/test/org/eclipse/papyrus/infra/core/sasheditor/pagesmodel/PanelTerm.java:[48]
-
[ERROR] * @see
org.eclipse.papyrus.infra.core.sasheditor.IModelObject.sashmodel.query.IQueryTerm#accept(org.eclipse.papyrus.infra.core.sasheditor.IPagesModelVisitor.sashmodel.query.IQueryVisitor,
org.eclipse.emf.ecore.EObject)
Theres
a lot of activity on the build
server right now, so it may
take time for any fix/commit
to be taken into account
Camille
...
and of course there is the
Gerrit process, with a lot of
contributions being delayed
for several weeks.
And
one of the main issues in the
diagrams is merging the
GMFGen/Generated code: weve
had a lot of regression due to
incorrect merge commits (We
dont have any tool to merge
the GMFGen model).
Diagrams
are really sensitive, thus the
M4/M5 refactorings, which will
hopefully help a lot in the
future. But for now... its
mostly regressions :) (If
these modifications hadnt
been pushed, we wouldnt even
be able to work on some
diagrams, and especially not
in parallel)
BTW,
if anyone can figure out
whats wrong with the
Master-Tests-Failures job...
it seems the Junit XML file is
corrupt, which breaks the
Publish Junit results and
breaks the build. I suppose
an invalid character appears
in one of the 10 000 tests of
Papyrus, but couldnt find
which one. So the job is now
suspended, and we cant really
use the test-annotations
framework on Master...
Ø
Interesting. Does SVG
require a native library that
isnt installed on the build
server? We really need to get
the tests green (IMHO before
M6). Is there something I can
do to help? (being quite
inexperienced with SVG and the
diagrams generally)
Batik/SVG
+ Apache Xerces. It looks like
the SaxParser Driver cant
be found. At least thats what
the Error log says. I suppose
its a plug-in dependency
issue (I remember we had
trouble with inconsistent
versions of Batik, because
they moved a class from one
plug-in to another, and in
some configurations, this
resulted in NoClassDefFound
errors). It may be another
occurrence of this issue, but
I havent really had the
opportunity to look into this
so far. Ive added an explicit
dependency to Xerces but this
didnt help; I didnt go much
further.
It
may also be a bug in Papyrus
(We had some case-sensitivity
errors in the images, which
worked fine in Windows/PDE
environment, but not in a
Linux/Installed eclipse
instance), but these errors
should be fixed now.
Camille
Ø
(everybody does run
the tests locally before
they push changes, right?
;-)
Sure
:) Remember this issue
about results being
different on Tycho and JDT
? :)
Thats
good. Its just that
sometimes it doesnt seem like
it. Ive been guilty, myself,
through overconfidence, so I
make liberal use of smilies.
:-)
A
lot of the test failures that
crop up from time to time are
reproducible locally, so they
mostly arent down to
differences in the build
environments. But hopefully
we are now a step closer to
improving that situation, too.
SVG
images seem to be failing
on the Hudson/Tycho build
for some reason. They
caused a lot of indirect
test failures (Took me 6
weeks to figure it out...
I guess the over-spammed
error log in the test
build didnt help). But
these tests were mostly
fine, when executed
locally.
Interesting.
Does SVG require a native
library that isnt installed
on the build server? We
really need to get the tests
green (IMHO before M6). Is
there something I can do to
help? (being quite
inexperienced with SVG and the
diagrams generally)