[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Neon.3 Update Problems: To Fix and How to Fix?

Fred,

I don’t know how to reproduce the update problems with these build artifacts yet. Where can I download an eclipse product  - or even better: can you share a download link that contains the EPP packages? With that I can run several manual tests and see how things turn out.

Marcel


On 25 Apr 2017, at 14:28, Frederic Gurr <frederic.gurr@xxxxxxxxxxx> wrote:

Thanks Jeff,

This is the comparison of the non-unique versions list:

neon3_respin aggregation build 20.04.
(https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/2/artifact/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt):

org.apache.httpcomponents.httpclient
4.3.6.v201411290715
4.5.2.v20161115-1643
4.3.6.v201511171540
4.2.6.v201311072007
org.apache.httpcomponents.httpcore
4.3.3.v201411290715
4.4.6.v20170210-0925
4.2.5.v201311072007

neon3_respin aggregation build 25.04.
(https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/3/artifact/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt):

org.apache.httpcomponents.httpclient
4.3.6.v201411290715
4.3.6.v201511171540
4.2.6.v201311072007
org.apache.httpcomponents.httpcore
4.3.3.v201411290715
4.2.5.v201311072007


So o.a.h.httpclient version 4.5.2 and o.a.h.httpcore version 4.4.6 are
not in the repo any more.

@All: Can someone confirm that this fixes the update problem (earlier
version to Neon.3)?

If it is confirmed to work, we have a fix for users that have not
upgraded to Neon.3 yet. Users that already upgraded are unaffected by
this. They still need to wait for a "wiring issue" fix.

Regards,

Fred


On 21.04.2017 23:00, Jeff Johnston wrote:
Just to confirm that the change has been merged to the Neon.3_respin branch.

-- Jeff J.

On Fri, Apr 21, 2017 at 4:11 PM, Jeff Johnston <jjohnstn@xxxxxxxxxx
<mailto:jjohnstn@xxxxxxxxxx>> wrote:

   I have done another respin.  In this case, I rebuilt the Linux Tools
   plug-ins to use the older version
   of docker-client and its dependencies which were used in Neon.2.  I
   have back-versioned the
   Docker tooling plug-ins to 2.3.0 with the current date.

   I have just pushed to gerrit for Neon.3_respin branch.

   https://git.eclipse.org/r/#/c/95501/
   <https://git.eclipse.org/r/#/c/95501/>

   If no-one objects, I will merge into the branch.  Verification is
   already successful.

   -- Jeff J.

   On Fri, Apr 21, 2017 at 11:50 AM, Jeff Johnston <jjohnstn@xxxxxxxxxx
   <mailto:jjohnstn@xxxxxxxxxx>> wrote:

       Marcel,

       Since the respin didn't fix the problem, we will try reverting
       the docker-client dependency and keeping as much of the added
       Neon.3 functionality as possible
       in place and cutting another point release.  I will let the list
       know when I have something in place.

       -- Jeff J.

       On Fri, Apr 21, 2017 at 6:38 AM, Daniel Megert
       <daniel_megert@xxxxxxxxxx <mailto:daniel_megert@xxxxxxxxxx>> wrote:

           Adding eclipse.org-planning-council@xxxxxxxxxxx
           <mailto:eclipse.org-planning-council@xxxxxxxxxxx> to this
           thread.

           Dani



           From:        "Dr. Marcel Bruch" <marcel.bruch@xxxxxxxxxxxxxx
           <mailto:marcel.bruch@xxxxxxxxxxxxxx>>
           To:        Cross project issues
           <cross-project-issues-dev@xxxxxxxxxxx
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
           Date:        21.04.2017 10:17
           Subject:        Re: [cross-project-issues-dev] Neon.3 Update
           Problems: To Fix and        How to Fix?
           Sent by:        cross-project-issues-dev-bounces@xxxxxxxxxxx
           <mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx>
           ------------------------------------------------------------------------



           Hi,

           I’ll briefly summarize the discussion we had at the AC
           yesterday:

           Given that we don’t know how the OSGI resolver will behave
           (even after Tom back-ported a fix to Neon) it would be
           preferred to just have the Apache HTTP*** versions in Neon.3
           that were already in Neon.2. This would to some extend
           “ensure" that we are "at least as as stable as Neon.2”. This
           would require us to rollback the changes that introduced the
           latest version of HTTPClient. As far as I know this would
           especially affect the Docker Tooling. (maybe more changes
           than that are needed)

           My question to the *Docker Tooling project lead*: Is it
           possible to rollback this last minute change and postpone it
           to Oxygen for the sake of making EGit, MPC, Oomph, USS, and
           Code Recommenders work reliably again - and giving us more
           trust that we won’t get into trouble with Neon.3? The
           simplest solution may be to contribute the docker tooling
           from Neon.2 in Neon.3. WDYT?

           Marcel




           On 20 Apr 2017, at 18:54, Frederic Gurr
           <_frederic.gurr@xxxxxxxxxxx_
           <mailto:frederic.gurr@xxxxxxxxxxx>> wrote:

           I can see
           org.apache.httpcomponents.httpclient_4.5.2.v20170210-0925 in
           /home/data/httpd/_download.eclipse.org/linuxtools/update-docker-2.3.1/plugins_
           <http://download.eclipse.org/linuxtools/update-docker-2.3.1/plugins>,
           but it's definitely not in the aggregated repo.


           On 20.04.2017 18:31, Jeff Johnston wrote:
           Fred,

           The version of httpclient also changed in our
           update-docker-2.3.1 repo from:

           org.apache.httpcomponents.httpclient_4.5.2.v20161115-1643

           to:

           org.apache.httpcomponents.httpclient_4.5.2.v20170210-0925

           Not sure why this change isn't being seen as well.

           -- Jeff J.



           On Thu, Apr 20, 2017 at 12:21 PM, Frederic Gurr
           <_frederic.gurr@xxxxxxxxxxx_
           <mailto:frederic.gurr@xxxxxxxxxxx><_mailto:frederic.gurr@eclipse.org_
           <mailto:frederic.gurr@xxxxxxxxxxx>>> wrote:

             Thanks Jeff,

             I ran a SimRel aggregation build. The only change I can
           see in the list
             of "Non Unique Versions used in repository" is that a
           different version
             of org.apache.httpcomponents.httpcore is now used. Instead of
             4.4.4.v20161115-1643 it's now 4.4.6.v20170210-0925.

             I compared

           _http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt_
           <http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt>

           <_http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt_
           <http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt>>
             and

           _https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt_
           <https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt>

           <_https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt_
           <https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt>>

             @All: is that the intended result?

             Regards,

             Fred


             On 19.04.2017 20:21, Jeff Johnston wrote:
           Hi Fred,

           I have just pushed a change to gerrit:
              _https://git.eclipse.org/r/#/c/95308/_
           <https://git.eclipse.org/r/#/c/95308/>
             <_https://git.eclipse.org/r/#/c/95308/_
           <https://git.eclipse.org/r/#/c/95308/>>

           I only changed the docker repository and left the other
           Linux Tools
           features alone
           since they were only bumped as part of the point release to
           fix the
           Docker Tooling plug-ins.

           I assume I can merge the patch if the gerrit verification is
           successful.  If this is wrong,
           let me know.

           -- Jeff J.

           On Wed, Apr 19, 2017 at 1:08 PM, Frederic Gurr
           <_frederic.gurr@xxxxxxxxxxx_
           <mailto:frederic.gurr@xxxxxxxxxxx><_mailto:frederic.gurr@eclipse.org_
           <mailto:frederic.gurr@xxxxxxxxxxx>>
              <_mailto:frederic.gurr@eclipse.org_
           <mailto:frederic.gurr@xxxxxxxxxxx>
             <_mailto:frederic.gurr@eclipse.org_
           <mailto:frederic.gurr@xxxxxxxxxxx>>>> wrote:

             Hi,

             Can you provide a patch for the SimRel build (branch
              "Neon.3_respin")
              that references the new version?

             Regards,

             Fred

             On 19.04.2017 17:27, Jeff Johnston wrote:
           Hi Ed,

           Linux tools spun a 5.3.1 release which now has a 2.3.1
              version of
              docker
           tooling.  The Linux tools download site has
              update-docker-2.3.1 and
           update-docker, both which have 2.3.1 versions of the docker.core
              plug-in
           and docker feature.  Not sure why you are not seeing this.

           -- Jeff J.

           On Wed, Apr 19, 2017 at 11:13 AM, Ed Merks
              <_ed.merks@xxxxxxxxx_
           <mailto:ed.merks@xxxxxxxxx><_mailto:ed.merks@gmail.com_
           <mailto:ed.merks@xxxxxxxxx>>
              <_mailto:ed.merks@gmail.com_<_mailto:ed.merks@gmail.com_
           <mailto:ed.merks@xxxxxxxxx>>>
           <_mailto:ed.merks@gmail.com_<_mailto:ed.merks@gmail.com_
           <mailto:ed.merks@xxxxxxxxx>>
              <_mailto:ed.merks@gmail.com_<_mailto:ed.merks@gmail.com_
           <mailto:ed.merks@xxxxxxxxx>>>>> wrote:

             Frederic,

             There seem to have been no notes/minutes taken during
              the meeting:



           _https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>
             <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>>

            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>

            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>>>

            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>

            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>>

            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>

            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
           <https://wiki.eclipse.org/Planning_Council/April_05_2017>>>>

             I recall agreeing to provide steps for reproducing the
              problem so
              that Thomas Watson could test if the wiring resolution
              fix he made
              for Oxygen also solves the problem for Neon.3.  The fact
              that he
              encountered "the mirroring problem" didn't help in that
              regard:

               _https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
              <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
           <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>>>

             In the end, he sent me a note saying (and I quote):

              I see that now there is the same number of
              httpcomponents bundles
              as there was in the messed up Oxygen M6 builds.  But
              here my back
              port of the resolver fix does not seem to have fixed
              the issue.
              I'm unsure if that is because it gave up with the sheer
              number of
              bundles or if something else is going wrong.  But at
              this point
              the backport of the resolver fix does not seem to be
              the solution
              to the problem.

             I assumed (wrongly I guess) that Thomas would
              investigate a more
              general fix to address the wiring problem.

             In the end, I also wasn't sure which version of the docker
              tools is
              proposed for contribution to Neon.3a.  I tried to search for
              update
              sites containing it like this:

             Nothing looks like a new version of 2.3.  Goodness knows
              where one
              should find what's being proposed for contribution...

             In any case, the proposed "solution" (A) really just
              changes the
              version of httpclient to be one that's not broken (missing
             packages), but it doesn't change the wiring problem in any
             fundamental way.  There will still be the four versions that
              can all
              be installed simultaneously, so we really should expect
              the same
              wiring problem(s).  In fact, I believe Oxygen M6 has
              effectively the
              same four httpcomponents.httpclient bundle as does
              Neon.3, so
              I'm a
              little suspicious whether the wiring problem is in fact
              really
              fixed
              even for Oxygen.  We won't know until M7 and that's a
              month away.
              It doesn't give me warm fuzzy feelings.

             So at this point it remains unclear the nature of the wiring
             problem(s).  Is it a bug? Is it fixable? Does the
              knowledge, will,
              and capacity to fix it exist?

             Without a fix to the wiring problem I think we can
              eliminate A
              as a
              solution, leaving B, C, and D (i.e., focus on problem
              avoidance
              approaches).  But I think if the wiring problem is a
              bug, it will
              come back, and it will raise its ugly head again when users
              install
              various technologies from various sources.  To my
              thinking, fixing
              the bug seems important.

             Regards,
             Ed


             On 19.04.2017 12:49, Frederic Gurr wrote:
              Hi Ed,

             In the last planning-council meeting you offered to
              evaluate
              if the
              fixed Linux Tools package works as expected and if
              there are
              still
              wiring issues.

             Can you give us an update on the current state?

             Regards,

             Fred

             On 31.03.2017 11:14, Ed Merks wrote:
              Hi,

             The original thread is fractured into many threads so its
              kind of
              impossible to follow each thread with a reply but I'll try
              at the bottom
              of this note, i.e., below the ===========

             But before doing that, I'd like to re-focus on the most
              important
              questions: *We currently have a problem with Neon.3,
              will we
              fix it, and
              if so how will we fix it?*

             The discussion has quickly digressed (constructively) into
              solving the
              issue of how Orbit dependencies should be managed by
              projects and by the
              release train.  Unfortunately I see this as a world hunger
              issue; not
              one that is easily addressed and I believe not one we can
              wait for in
              order to solve the Neon.3 problem.  Let's face it,
              we've not
              been able
              to produce a proper Oxygen milestone in months, we still
              don't have one
              now, and we won't have one until next month, we hope.

             For Neon we've done three maintenance releases. Neon.1
              needed a respin
              and Neon.3 looks to be in need of the same thing. Clearly
              something is
              seriously wrong.  But if we spend our time on solving the
              Orbit world
              hunger issue, will we arrive at a solution in time for
              Oxygen, let alone
              in time to fix Neon.3?  I am very, very doubtful.

             As another data point, if I install the
              egg-laying-wool-milk-pig for
              Neon.3.  The following happens.  I'm prompted to
              accept this
              license:

                 Red Hat, Inc. licenses these features and plugins
              to you
              under
                  certain open source licenses (or aggregations of such
              licenses),
                  which in a particular case may include the Eclipse
              Public License,
                  the GNU Lesser General Public License, and/or certain
              other open
                  source licenses. For precise licensing details,
              consult the
                  corresponding source code, or contact Red Hat,
              Attn: General
                  Counsel, 100 East Davie St., Raleigh NC 27601 USA.

             I'm not sure how this license slipped into the release
              train.   Aren't
              there checks for this?  (Sorry to digress, but this is
              also
              unacceptable.)

             Launching the final installation comes up like this:

             Clearly a disgusting mess, but I've mentioned that before
              and the same
              projects are still doing the same bad things, so we
              clearly
              all accept
              this situation as normal.

             The most important point here is the error log (first
              attachment) is
              full of exactly the problem indications (bundle wiring
              problems) we
              should have expected from the Neon.3 repository's
              contents,
              if someone
              were to install an arbitrary combination of the
              repository's
              contents.
              It's really not so hard to test this!

             If I create the same installation with my local build
              of the
              Oomph 1.8
              installer---which installs my locally built version of
              Oomph
              1.8 so the
              Oomph setup plugins are no longer disabled because I
              made the
              userstorage dependency optional and eliminated the strict
              <=4.4 upper
              bound constraints on httpclient, which was such a bad
              idea I
              can almost
              have a canary to think this done to solve a problem
              with no
              anticipation
              of the problems it would cause---then I can visit all the
              preference
              pages producing the second attached much larger log. It
              seems clear
              that proper testing really doesn't happen for far too many
              projects on
              the train.  With distributed responsibility, no one is
              really responsible...

             ==================================

             Orbit Issues

             1) Respinning Linux Tools against Oxygen Mx seems to miss
              the point that
              we should only distribute released versions of
              bundles,  so
              no Neon
              build should redistribute any unreleased version of
              anything.  If a new
              version of something is needed for security reasons or
              other
              reasons, it
              should be released first.  And doing that in a maintenance
              train without
              testing the overall impact is clearly something we should
              never do again
              (without waving a bunch of red flags of warning). And
              as Martin
              Oberhuber asks, is nothing in place to check for this?  So
              suppose we do
              respin with a fixed released version, like what we
              have for
              Oxygen M6,
              then most likely we'd still have the problems we have in
              Oxygen M6 so
              we'd need a fix to the resolver in Neon.  Better would
              seem
              to respin
              with the old version(s) of the Orbit bundles, but
              somehow we
              can never
              delete the broken version from Neon and because it has a
              higher version
              number is likely to slip back in unexpected (though
              hopefully not, given
              that features have pinned their bundle versions).

             2) Don't include Orbit bundles in your project's features.
              Sounds like
              a great idea, but begs endless questions, and while
              solving
              a problem
              might well introduce more new problems than it
              solves.  The
              first
              question (as Carsten points out) is how do these
              things end
              up in a
              repository, and if they are in a repository somehow,
              how are
              they
              categorized?  It's hard to get them in and once you
              do, they're
              categorized poorly.  The next question is, how do they end
              up in the
              release train, if the projects that need them don't
              contribute them?
              Directly from Orbit you say?  But which ones should be
              pulled in from
              Orbit and how is that discovered?   Are those the ones the
              projects have
              tested against? Then there is the question of whether an
              installation is
              deterministic if the bundle version isn't pinned?
              It's not;
              it will
              depend on what's in the repos that are available at
              resolve
              time.  But
              Gunnar argues that even packages are not deterministic,
              which I think is
              false: if the feature pins the bundle version and the
              package requires
              the feature, then the pinned bundle is definitely in that
              package.  But
              regardless, Gunnar's important point is that the runtime
              wiring seems
              kind of non-determinstic, and while uses constraints might
              help, who the
              heck understands those well, what tooling produces it
              correctly for us,
              is that nicely integrated in PDE, and will it be properly
              maintained (in
              contrast to lower bound constraints which you can pretty
              expect will
              remain on whatever stale version they were initially set
              to).  This may
              well be the right direction in which to go, but getting
              there isn't
              going to be even half the fun...

             Regards,
             Ed





             _______________________________________________
             cross-project-issues-dev mailing list
             _cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>>
              To change your delivery options, retrieve your
              password, or
              unsubscribe from this list, visit



            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>



            <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>>

              _______________________________________________
             cross-project-issues-dev mailing list
             _cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>>
              To change your delivery options, retrieve your password, or
              unsubscribe from this list, visit



            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>



            <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>>


             _______________________________________________
             cross-project-issues-dev mailing list
             _cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>>
              To change your delivery options, retrieve your password, or
             unsubscribe from this list, visit



            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>



            <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>>




           _______________________________________________
           cross-project-issues-dev mailing list_
           __cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
           To change your delivery options, retrieve your password, or
              unsubscribe from this list, visit


            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>

              _______________________________________________
             cross-project-issues-dev mailing list
             _cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
              To change your delivery options, retrieve your password, or
             unsubscribe from this list, visit


           _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>


           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>




           _______________________________________________
           cross-project-issues-dev mailing list_
           __cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
              <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
           To change your delivery options, retrieve your password, or
              unsubscribe from this list, visit
           _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

            <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>

              _______________________________________________
             cross-project-issues-dev mailing list
             _cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
             <_mailto:cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
             To change your delivery options, retrieve your password, or
             unsubscribe from this list, visit

           _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>




           _______________________________________________
           cross-project-issues-dev mailing list_
           __cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
           To change your delivery options, retrieve your password, or
           unsubscribe from this list, visit_
           __https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           _______________________________________________
           cross-project-issues-dev mailing list_
           __cross-project-issues-dev@xxxxxxxxxxx_
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
           To change your delivery options, retrieve your password, or
           unsubscribe from this list, visit_
           __https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>

           --
           Codetrails GmbH
           The knowledge transfer company

           Robert-Bosch-Str. 7, 64293 Darmstadt
           Phone: +49-6151-276-7092 <tel:+49%206151%202767092>
           Mobile: +49-179-131-7721 <tel:+49%20179%201317721>_
           __http://www.codetrails.com/_

           Managing Director: Dr. Marcel Bruch
           Handelsregister: Darmstadt HRB 91940
           _______________________________________________
           cross-project-issues-dev mailing list
           cross-project-issues-dev@xxxxxxxxxxx
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
           To change your delivery options, retrieve your password, or
           unsubscribe from this list, visit
           https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>



           _______________________________________________
           cross-project-issues-dev mailing list
           cross-project-issues-dev@xxxxxxxxxxx
           <mailto:cross-project-issues-dev@xxxxxxxxxxx>
           To change your delivery options, retrieve your password, or
           unsubscribe from this list, visit
           https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev
           <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>






_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev


_______________________________________________
cross-project-issues-dev mailing list
cross-project-issues-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev

-- 
Codetrails GmbH
The knowledge transfer company

Robert-Bosch-Str. 7, 64293 Darmstadt
Phone: +49-6151-276-7092
Mobile: +49-179-131-7721
http://www.codetrails.com/

Managing Director: Dr. Marcel Bruch
Handelsregister: Darmstadt HRB 91940