[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [mylyn-dev] continuous testing
|
Steffen,
To clarify, it is not just about optimal. There been CVS commits
(supposedly driven by these external tasks) that break Mylyn dev
guidelines [1] because they didn't have corresponding Bugzilla task
and/or associated task context. Maybe if current system can't work with
Bugzilla, perhaps there should be some manual step to bring these tasks
back to Bugzilla.
Support for multiple task repositories for given resource in Mylyn
maybe is too exotic solution to this issue and from my knowledge of
repository linking support in Mylyn, it would cause an intrusive API
change. But if decided this stuff should be implemented before hand, so
no task context info and linking from the change sets will be broken.
regards,
Eugene
[1] http://wiki.eclipse.org/Mylyn_Contributor_Reference#Contributors
Steffen Pingel wrote:
Eugene, thank you for your feedback. There are certainly numerous improvements
that can be made to the current aproach for continuous testing. The setup is
still work in progress. I aimed at getting the the maximum value by spending
the minimal amout of my spare time. Now that it is up and running we can get
some mileage and evaluate if this will provide enough value to make it part
of our development process.
I agree with your concern that it is not optimal to track the test results
separately from the Eclipse Bugzilla. At this point the tool I created for
reporting the test results is by no means sufficiently tested or optimized to
publish to a production repository though. It would require significantly
more work to harden the tool and permission from the Eclipse webmasters etc.
to do that.
Contexts can be easily attached to the tickets that store the test results and
shared that way. Commit messages should point back to the original Bugzilla
report that is related to the cause of the test failure. This workflow might
impose additional overhead for committers but it will also help us learn more
about using multiple task repositories for a single source base and optimize
the tooling around that use case if necessary.
Please don't be discouraged by the fact that the orignial bug 213100 is marked
as resolved. I will read your comments and respond appropriately. If the
discussion brings up ideas that are feasible and there is time to pursue them
we can create additional bug reports.
Steffen
On Wednesday 19 December 2007, Eugene Kuleshov wrote:
Steffen,
After looking at this for a few days I think there are some problems
with this approach that need to be addressed somehow. I've added few
comments to the suggested bug report, though it is odd, because report
is already closed as resolved.
Anyways, the biggest issue is that it introduced second source of
issues that is participating in the code changes (in other words change
sets and CVS commits, history, annotations, etc), which is not listed as
the project linked issue tracker. So we are losing contextual
information about these changes, including the task contexts.
I am not sure what would be the right way to resolve this, but maybe
continuous testing tool should use project Bugzilla for all reporting.
regards,
Eugene
Steffen Pingel wrote:
It's in CVS in sandbox/org.eclipse.mylyn.test.report. I'll add some
documentation about how to run it in the next few days.
Steffen
On Sunday 16 December 2007, Eugene Kuleshov wrote:
Great stuff Steffen! I wonder if code for your headless application
available? It would be interesting to know how all the pieces are glued
together.
Thanks
Eugene
Steffen Pingel wrote:
I am in the process of setting up a continuous build process that will
run the Mylyn AllTests suite and AllPerformanceTest suite nightly. I
have implemented a headless application based on Mylyn that publishes
test results to a Trac repository:
http://mylyn.eclipse.org/bitten/
Every open ticket corresponds to a failing test in the latest nightly
build. The last comment on a ticket describes the cause of the test
failure including a stack trace. Once a test succeeds the ticket is
closed. It is reopened if a test fails during a future build.
Monitoring tests is easy by creating a query against the repository
that includes all tickets (the repository allows anonymous access).
Please note that some of the tests failed due to running in a deployed
environment.
The goal is to make this a seamless part of the Mylyn development
workflow to help us avoid having failed tests go unnoticed. Please
comment on this bug for feedback and suggestions:
213100: automate publishing of failed tests
https://bugs.eclipse.org/bugs/show_bug.cgi?id=213100
Steffen
_______________________________________________
mylyn-dev mailing list
mylyn-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/mylyn-dev
_______________________________________________
mylyn-dev mailing list
mylyn-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/mylyn-dev
