Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cross-project-issues-dev] Neon.3 Update Problems: To Fix and How to Fix?

Should we add uninstallbundle commands in the p2.inf file for the Docker Tooling feature to remove all the http bundles
that might have been installed by a previous version?  I have found another problem when installing the Docker tooling
feature from the new aggregation on top of the Neon.3 EPP for Eclipse committers.  Multiple slf4j bundles cause a gradle failure.
I am working on a fix now.  I can add the p2.inf stuff at the same time if deemed useful.

-- Jeff J.

On Tue, Apr 25, 2017 at 11:19 AM, Ed Merks <ed.merks@xxxxxxxxx> wrote:
Fred,

No I didn't try that.  It's much harder to try that.  Oomph itself is broken, so I can't run setup tasks.  Check for Updates finds no updates because I installed all the categories and their IUs don't have updated versions.   I tried explicitly installing Docker so that it would do an update, but that didn't not solve the wiring problem.  I also running the p2 task in an IDE with Oomph 1.8 where I can run setup tasks (because Oomph's dependency on userstorage is optional in 1.8) and where the wiring problems also occurred so that more things got updated, but this also did not resolve the wiring problem.  So it's not clear if updates to an already -broken IDE will fix that IDE; it doesn't look so promising, though I don't understand why that wouldn't work.   If nothing requires a given bundle, I would expect that bundle to be removed from the profile, but I don't see the problematic bundles being removed...

Regards,
Ed


On 25.04.2017 16:41, Frederic Gurr wrote:
Thanks Ed.

That sounds promising. Did you also test updating a broken ELWMS with
the new repo?

I'm currently trying to reproduce the error with EPP packages, but so
far updating a Neon.2 package to Neon.3 (with "Check for Updates") did
not result in a broken MPC. Not sure what I'm missing.

Regards,

Fred

On 25.04.2017 16:30, Ed Merks wrote:
Fred,

Installing the Eierlegende Wollmilchsau with this additional repository
in the p2 task's repository list results in an installation without
wiring problems.

The profile of the installation contains version 2.3.0.20170421191 of
org.eclipse.linuxtools.docker.core with is indeed the version available
in
https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/3/artifact/aggregation/final
so I think this is a good demonstration that installations creation from
this repository's contents will not have the wiring problems we've been
seeing in Neon.3.

Regards,
Ed


Ed,

The temporary update site URL is:

https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/3/artifact/aggregation/final


Regards,

Fred

On 25.04.2017 15:13, Ed Merks wrote:
Fred,

What update site URL contains these results?

Regards,
Ed


On 25.04.2017 14:28, Frederic Gurr wrote:
Thanks Jeff,

This is the comparison of the non-unique versions list:

neon3_respin aggregation build 20.04.
(https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/2/artifact/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt):



org.apache.httpcomponents.httpclient
      4.3.6.v201411290715
      4.5.2.v20161115-1643
      4.3.6.v201511171540
      4.2.6.v201311072007
org.apache.httpcomponents.httpcore
      4.3.3.v201411290715
      4.4.6.v20170210-0925
      4.2.5.v201311072007
      neon3_respin aggregation build 25.04.
(https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/3/artifact/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt):



org.apache.httpcomponents.httpclient
      4.3.6.v201411290715
      4.3.6.v201511171540
      4.2.6.v201311072007
org.apache.httpcomponents.httpcore
      4.3.3.v201411290715
      4.2.5.v201311072007


So o.a.h.httpclient version 4.5.2 and o.a.h.httpcore version 4.4.6 are
not in the repo any more.

@All: Can someone confirm that this fixes the update problem (earlier
version to Neon.3)?

If it is confirmed to work, we have a fix for users that have not
upgraded to Neon.3 yet. Users that already upgraded are unaffected by
this. They still need to wait for a "wiring issue" fix.

Regards,

Fred


On 21.04.2017 23:00, Jeff Johnston wrote:
Just to confirm that the change has been merged to the Neon.3_respin
branch.

-- Jeff J.

On Fri, Apr 21, 2017 at 4:11 PM, Jeff Johnston <jjohnstn@xxxxxxxxxx
<mailto:jjohnstn@xxxxxxxxxx>> wrote:

       I have done another respin.  In this case, I rebuilt the Linux
Tools
       plug-ins to use the older version
       of docker-client and its dependencies which were used in
Neon.2.  I
       have back-versioned the
       Docker tooling plug-ins to 2.3.0 with the current date.

       I have just pushed to gerrit for Neon.3_respin branch.

       https://git.eclipse.org/r/#/c/95501/
       <https://git.eclipse.org/r/#/c/95501/>

       If no-one objects, I will merge into the branch.
Verification is
       already successful.

       -- Jeff J.

       On Fri, Apr 21, 2017 at 11:50 AM, Jeff Johnston
<jjohnstn@xxxxxxxxxx
       <mailto:jjohnstn@xxxxxxxxxx>> wrote:

           Marcel,

           Since the respin didn't fix the problem, we will try
reverting
           the docker-client dependency and keeping as much of the
added
           Neon.3 functionality as possible
           in place and cutting another point release.  I will let the
list
           know when I have something in place.

           -- Jeff J.

           On Fri, Apr 21, 2017 at 6:38 AM, Daniel Megert
           <daniel_megert@xxxxxxxxxx <mailto:daniel_megert@xxxxxx.com>>
wrote:

               Adding eclipse.org-planning-council@eclipse.org
               <mailto:eclipse.org-planning-council@xxxxxxxxxxx> to
this
               thread.

               Dani



               From:        "Dr. Marcel Bruch"
<marcel.bruch@xxxxxxxxxxxxxx
               <mailto:marcel.bruch@codetrails.com>>
               To:        Cross project issues
               <cross-project-issues-dev@eclipse.org
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
               Date:        21.04.2017 10:17
               Subject:        Re: [cross-project-issues-dev] Neon.3
Update
               Problems: To Fix and        How to Fix?
               Sent by:
cross-project-issues-dev-bounces@xxxxxxxxxxx
               <mailto:cross-project-issues-dev-bounces@xxxxxxxxxxx>
             ------------------------------------------------------------------------




               Hi,

               I’ll briefly summarize the discussion we had at the AC
               yesterday:

               Given that we don’t know how the OSGI resolver will
behave
               (even after Tom back-ported a fix to Neon) it would be
               preferred to just have the Apache HTTP*** versions in
Neon.3
               that were already in Neon.2. This would to some extend
               “ensure" that we are "at least as as stable as Neon.2”.
This
               would require us to rollback the changes that introduced
the
               latest version of HTTPClient. As far as I know this
would
               especially affect the Docker Tooling. (maybe more
changes
               than that are needed)

               My question to the *Docker Tooling project lead*: Is it
               possible to rollback this last minute change and
postpone it
               to Oxygen for the sake of making EGit, MPC, Oomph,
USS, and
               Code Recommenders work reliably again - and giving us
more
               trust that we won’t get into trouble with Neon.3? The
               simplest solution may be to contribute the docker
tooling
               from Neon.2 in Neon.3. WDYT?

               Marcel




               On 20 Apr 2017, at 18:54, Frederic Gurr
               <_frederic.gurr@eclipse.org_
               <mailto:frederic.gurr@eclipse.org>> wrote:

               I can see
             org.apache.httpcomponents.httpclient_4.5.2.v20170210-0925 in
             /home/data/httpd/_download.eclipse.org/linuxtools/update-docker-2.3.1/plugins_


             <http://download.eclipse.org/linuxtools/update-docker-2.3.1/plugins>,
               but it's definitely not in the aggregated repo.


               On 20.04.2017 18:31, Jeff Johnston wrote:
               Fred,

               The version of httpclient also changed in our
               update-docker-2.3.1 repo from:

              org.apache.httpcomponents.httpclient_4.5.2.v20161115-1643

               to:

              org.apache.httpcomponents.httpclient_4.5.2.v20170210-0925

               Not sure why this change isn't being seen as well.

               -- Jeff J.



               On Thu, Apr 20, 2017 at 12:21 PM, Frederic Gurr
               <_frederic.gurr@eclipse.org_
             <mailto:frederic.gurr@eclipse.org><_mailto:frederic.gurr@eclipse.org_
               <mailto:frederic.gurr@eclipse.org>>> wrote:

                 Thanks Jeff,

                 I ran a SimRel aggregation build. The only change I
can
               see in the list
                 of "Non Unique Versions used in repository" is that a
               different version
                 of org.apache.httpcomponents.httpcore is now used.
Instead of
                 4.4.4.v20161115-1643 it's now 4.4.6.v20170210-0925.

                 I compared
                           _http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt_


             <http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt>


                           <_http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt_


             <http://download.eclipse.org/releases/neon/201703231000/buildInfo/reporeports/reports/nonUniqueVersions.txt>>


                 and
                           _https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt_


             <https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt>


                           <_https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt_


             <https://hudson.eclipse.org/simrel/job/simrel.neon.3_respin.runaggregator.BUILD__CLEAN/ws/aggregation/final/buildInfo/reporeports/reports/nonUniqueVersions.txt>>



                 @All: is that the intended result?

                 Regards,

                 Fred


                 On 19.04.2017 20:21, Jeff Johnston wrote:
               Hi Fred,

               I have just pushed a change to gerrit:
                  _https://git.eclipse.org/r/#/c/95308/_
               <https://git.eclipse.org/r/#/c/95308/>
                 <_https://git.eclipse.org/r/#/c/95308/_
               <https://git.eclipse.org/r/#/c/95308/>>

               I only changed the docker repository and left the other
               Linux Tools
               features alone
               since they were only bumped as part of the point
release to
               fix the
               Docker Tooling plug-ins.

               I assume I can merge the patch if the gerrit
verification is
               successful.  If this is wrong,
               let me know.

               -- Jeff J.

               On Wed, Apr 19, 2017 at 1:08 PM, Frederic Gurr
               <_frederic.gurr@eclipse.org_
             <mailto:frederic.gurr@eclipse.org><_mailto:frederic.gurr@eclipse.org_
               <mailto:frederic.gurr@eclipse.org>>
                  <_mailto:frederic.gurr@eclipse.org_
               <mailto:frederic.gurr@eclipse.org>
                 <_mailto:frederic.gurr@eclipse.org_
               <mailto:frederic.gurr@eclipse.org>>>> wrote:

                 Hi,

                 Can you provide a patch for the SimRel build (branch
                  "Neon.3_respin")
                  that references the new version?

                 Regards,

                 Fred

                 On 19.04.2017 17:27, Jeff Johnston wrote:
               Hi Ed,

               Linux tools spun a 5.3.1 release which now has a 2.3.1
                  version of
                  docker
               tooling.  The Linux tools download site has
                  update-docker-2.3.1 and
               update-docker, both which have 2.3.1 versions of the
docker.core
                  plug-in
               and docker feature.  Not sure why you are not seeing
this.

               -- Jeff J.

               On Wed, Apr 19, 2017 at 11:13 AM, Ed Merks
                  <_ed.merks@gmail.com_
               <mailto:ed.merks@xxxxxxxxx><_mailto:ed.merks@gmail.com_
               <mailto:ed.merks@xxxxxxxxx>>
                 <_mailto:ed.merks@gmail.com_<_mailto:ed.merks@gmail.com_
               <mailto:ed.merks@xxxxxxxxx>>>
               <_mailto:ed.merks@gmail.com_<_mailto:ed.merks@gmail.com_
               <mailto:ed.merks@xxxxxxxxx>>
                 <_mailto:ed.merks@gmail.com_<_mailto:ed.merks@gmail.com_
               <mailto:ed.merks@xxxxxxxxx>>>>> wrote:

                 Frederic,

                 There seem to have been no notes/minutes taken during
                  the meeting:


                             _https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>
               <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>>
                            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>
                            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>>>
                            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>
                            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>>
                            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
              <https://wiki.eclipse.org/Planning_Council/April_05_2017>
                            <_https://wiki.eclipse.org/Planning_Council/April_05_2017_
             <https://wiki.eclipse.org/Planning_Council/April_05_2017>>>>

                 I recall agreeing to provide steps for reproducing the
                  problem so
                  that Thomas Watson could test if the wiring
resolution
                  fix he made
                  for Oxygen also solves the problem for Neon.3.
The fact
                  that he
                  encountered "the mirroring problem" didn't help in
that
                  regard:

                  _https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
               <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>
                 <_https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213_
              <https://bugs.eclipse.org/bugs/show_bug.cgi?id=515213>>>>

                 In the end, he sent me a note saying (and I quote):

                  I see that now there is the same number of
                  httpcomponents bundles
                  as there was in the messed up Oxygen M6 builds.  But
                  here my back
                  port of the resolver fix does not seem to have fixed
                  the issue.
                  I'm unsure if that is because it gave up with the
sheer
                  number of
                  bundles or if something else is going wrong.  But at
                  this point
                  the backport of the resolver fix does not seem to be
                  the solution
                  to the problem.

                 I assumed (wrongly I guess) that Thomas would
                  investigate a more
                  general fix to address the wiring problem.

                 In the end, I also wasn't sure which version of the
docker
                  tools is
                  proposed for contribution to Neon.3a.  I tried to
search for
                  update
                  sites containing it like this:

                 Nothing looks like a new version of 2.3.  Goodness
knows
                  where one
                  should find what's being proposed for contribution...

                 In any case, the proposed "solution" (A) really just
                  changes the
                  version of httpclient to be one that's not broken
(missing
                 packages), but it doesn't change the wiring problem in
any
                 fundamental way.  There will still be the four
versions that
                  can all
                  be installed simultaneously, so we really should
expect
                  the same
                  wiring problem(s).  In fact, I believe Oxygen M6 has
                  effectively the
                  same four httpcomponents.httpclient bundle as does
                  Neon.3, so
                  I'm a
                  little suspicious whether the wiring problem is in
fact
                  really
                  fixed
                  even for Oxygen.  We won't know until M7 and that's a
                  month away.
                  It doesn't give me warm fuzzy feelings.

                 So at this point it remains unclear the nature of the
wiring
                 problem(s).  Is it a bug? Is it fixable? Does the
                  knowledge, will,
                  and capacity to fix it exist?

                 Without a fix to the wiring problem I think we can
                  eliminate A
                  as a
                  solution, leaving B, C, and D (i.e., focus on problem
                  avoidance
                  approaches).  But I think if the wiring problem is a
                  bug, it will
                  come back, and it will raise its ugly head again when
users
                  install
                  various technologies from various sources.  To my
                  thinking, fixing
                  the bug seems important.

                 Regards,
                 Ed


                 On 19.04.2017 12:49, Frederic Gurr wrote:
                  Hi Ed,

                 In the last planning-council meeting you offered to
                  evaluate
                  if the
                  fixed Linux Tools package works as expected and if
                  there are
                  still
                  wiring issues.

                 Can you give us an update on the current state?

                 Regards,

                 Fred

                 On 31.03.2017 11:14, Ed Merks wrote:
                  Hi,

                 The original thread is fractured into many threads
so its
                  kind of
                  impossible to follow each thread with a reply but
I'll try
                  at the bottom
                  of this note, i.e., below the ===========

                 But before doing that, I'd like to re-focus on the
most
                  important
                  questions: *We currently have a problem with Neon.3,
                  will we
                  fix it, and
                  if so how will we fix it?*

                 The discussion has quickly digressed (constructively)
into
                  solving the
                  issue of how Orbit dependencies should be managed by
                  projects and by the
                  release train.  Unfortunately I see this as a world
hunger
                  issue; not
                  one that is easily addressed and I believe not one we
can
                  wait for in
                  order to solve the Neon.3 problem.  Let's face it,
                  we've not
                  been able
                  to produce a proper Oxygen milestone in months, we
still
                  don't have one
                  now, and we won't have one until next month, we hope.

                 For Neon we've done three maintenance releases. Neon.1
                  needed a respin
                  and Neon.3 looks to be in need of the same thing.
Clearly
                  something is
                  seriously wrong.  But if we spend our time on solving
the
                  Orbit world
                  hunger issue, will we arrive at a solution in time
for
                  Oxygen, let alone
                  in time to fix Neon.3?  I am very, very doubtful.

                 As another data point, if I install the
                  egg-laying-wool-milk-pig for
                  Neon.3.  The following happens.  I'm prompted to
                  accept this
                  license:

                     Red Hat, Inc. licenses these features and plugins
                  to you
                  under
                      certain open source licenses (or aggregations of
such
                  licenses),
                      which in a particular case may include the
Eclipse
                  Public License,
                      the GNU Lesser General Public License, and/or
certain
                  other open
                      source licenses. For precise licensing details,
                  consult the
                      corresponding source code, or contact Red Hat,
                  Attn: General
                      Counsel, 100 East Davie St., Raleigh NC 27601
USA.

                 I'm not sure how this license slipped into the release
                  train.   Aren't
                  there checks for this?  (Sorry to digress, but
this is
                  also
                  unacceptable.)

                 Launching the final installation comes up like this:

                 Clearly a disgusting mess, but I've mentioned that
before
                  and the same
                  projects are still doing the same bad things, so we
                  clearly
                  all accept
                  this situation as normal.

                 The most important point here is the error log (first
                  attachment) is
                  full of exactly the problem indications (bundle
wiring
                  problems) we
                  should have expected from the Neon.3 repository's
                  contents,
                  if someone
                  were to install an arbitrary combination of the
                  repository's
                  contents.
                  It's really not so hard to test this!

                 If I create the same installation with my local build
                  of the
                  Oomph 1.8
                  installer---which installs my locally built
version of
                  Oomph
                  1.8 so the
                  Oomph setup plugins are no longer disabled because I
                  made the
                  userstorage dependency optional and eliminated the
strict
                  <=4.4 upper
                  bound constraints on httpclient, which was such a bad
                  idea I
                  can almost
                  have a canary to think this done to solve a problem
                  with no
                  anticipation
                  of the problems it would cause---then I can visit all
the
                  preference
                  pages producing the second attached much larger
log. It
                  seems clear
                  that proper testing really doesn't happen for far too
many
                  projects on
                  the train.  With distributed responsibility, no
one is
                  really responsible...

                 ==================================

                 Orbit Issues

                 1) Respinning Linux Tools against Oxygen Mx seems
to miss
                  the point that
                  we should only distribute released versions of
                  bundles,  so
                  no Neon
                  build should redistribute any unreleased version of
                  anything.  If a new
                  version of something is needed for security
reasons or
                  other
                  reasons, it
                  should be released first.  And doing that in a
maintenance
                  train without
                  testing the overall impact is clearly something we
should
                  never do again
                  (without waving a bunch of red flags of warning). And
                  as Martin
                  Oberhuber asks, is nothing in place to check for
this?  So
                  suppose we do
                  respin with a fixed released version, like what we
                  have for
                  Oxygen M6,
                  then most likely we'd still have the problems we
have in
                  Oxygen M6 so
                  we'd need a fix to the resolver in Neon.  Better
would
                  seem
                  to respin
                  with the old version(s) of the Orbit bundles, but
                  somehow we
                  can never
                  delete the broken version from Neon and because it
has a
                  higher version
                  number is likely to slip back in unexpected (though
                  hopefully not, given
                  that features have pinned their bundle versions).

                 2) Don't include Orbit bundles in your project's
features.
                  Sounds like
                  a great idea, but begs endless questions, and while
                  solving
                  a problem
                  might well introduce more new problems than it
                  solves.  The
                  first
                  question (as Carsten points out) is how do these
                  things end
                  up in a
                  repository, and if they are in a repository somehow,
                  how are
                  they
                  categorized?  It's hard to get them in and once you
                  do, they're
                  categorized poorly.  The next question is, how do
they end
                  up in the
                  release train, if the projects that need them don't
                  contribute them?
                  Directly from Orbit you say?  But which ones
should be
                  pulled in from
                  Orbit and how is that discovered?   Are those the
ones the
                  projects have
                  tested against? Then there is the question of
whether an
                  installation is
                  deterministic if the bundle version isn't pinned?
                  It's not;
                  it will
                  depend on what's in the repos that are available at
                  resolve
                  time.  But
                  Gunnar argues that even packages are not
deterministic,
                  which I think is
                  false: if the feature pins the bundle version and the
                  package requires
                  the feature, then the pinned bundle is definitely in
that
                  package.  But
                  regardless, Gunnar's important point is that the
runtime
                  wiring seems
                  kind of non-determinstic, and while uses constraints
might
                  help, who the
                  heck understands those well, what tooling produces it
                  correctly for us,
                  is that nicely integrated in PDE, and will it be
properly
                  maintained (in
                  contrast to lower bound constraints which you can
pretty
                  expect will
                  remain on whatever stale version they were
initially set
                  to).  This may
                  well be the right direction in which to go, but
getting
                  there isn't
                  going to be even half the fun...

                 Regards,
                 Ed





                 _______________________________________________
                 cross-project-issues-dev mailing list
                 _cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>>
                  To change your delivery options, retrieve your
                  password, or
                  unsubscribe from this list, visit


                            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>

                             <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>


                            <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>

                             <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>>

                  _______________________________________________
                 cross-project-issues-dev mailing list
                 _cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>>
                  To change your delivery options, retrieve your
password, or
                  unsubscribe from this list, visit


                            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>

                             <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>


                            <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>

                             <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>>>


                 _______________________________________________
                 cross-project-issues-dev mailing list
                 _cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>
                  <_mailto:cross-project-issues-dev@eclipse.org_
               <mailto:cross-project-issues-dev@xxxxxxxxxxx>>>>
                  To change your delivery options, retrieve your
password, or
                 unsubscribe from this list, visit


                            _https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>
                           <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_
             <https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev>>

                             <_https://dev.eclipse.org/mailman/listinfo/cross-project-issues-dev_


Back to the top