Bug 551147 - Unable to use I20190916-1800 build due cycle errors
Summary: Unable to use I20190916-1800 build due cycle errors
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Resources (show other bugs)
Version: 4.14   Edit
Hardware: All All
: P3 blocker (vote)
Target Milestone: 4.14 M1   Edit
Assignee: Andrey Loskutov CLA
QA Contact:
URL:
Whiteboard:
Keywords: regression
Depends on:
Blocks:
 
Reported: 2019-09-17 03:19 EDT by Andrey Loskutov CLA
Modified: 2019-09-23 05:00 EDT (History)
9 users (show)

See Also:


Attachments
PDE dependencies (658.32 KB, image/png)
2019-09-18 14:06 EDT, Lars Vogel CLA
no flags Details
PDE dependencies new (230.56 KB, image/png)
2019-09-18 14:14 EDT, Lars Vogel CLA
no flags Details
duplicated bundles (108.50 KB, image/png)
2019-09-22 03:50 EDT, Andrey Loskutov CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Andrey Loskutov CLA 2019-09-17 03:19:08 EDT
I've switched from SDK-I20190915-1800 to I20190916-1800 and got multiple "cycle errors" after full build. I can't get rid of those errors. After many close/open/clean build attempts the errors disappear, but appear again on startup.

A cycle was detected in the build path of project 'org.eclipse.jdt.debug'. The cycle consists of projects {org.eclipse.jdt.compiler.apt, External Plug-in Libraries, org.eclipse.jdt.apt.pluggable.core, org.eclipse.jdt.astview, org.eclipse.jdt.ui, org.eclipse.jdt.launching, org.eclipse.jdt.debug, org.eclipse.jdt.core.manipulation, org.eclipse.jdt.debug.ui, org.eclipse.jdt.junit, org.eclipse.jdt.junit.core}.

I don't see any changes except runtime changes via bug 550738 and bug 550038 and various changes from Lars in rt.equinox.p2 which could affect resolution (bug 512678, bug 550462).
Comment 1 Andrey Loskutov CLA 2019-09-17 03:36:03 EDT
(In reply to Andrey Loskutov from comment #0)
> I've switched from SDK-I20190915-1800 to I20190916-1800 and got multiple
> "cycle errors" after full build. I can't get rid of those errors. After many
> close/open/clean build attempts the errors disappear, but appear again on
> startup.

OK, on a *third* (!?!) restart and numerous other rebuilds, JDT index cleaning, adding/removing all plugins to the workspace etc all the cycle errors disappeared.

I've spent ~1 hour to get the workspace to an almost error free state (bug 551109 error is still there). I haven't seen such kind of issue for a long time now, and I update to nightly builds every week for some years.
Comment 2 Lars Vogel CLA 2019-09-17 13:45:46 EDT
Cannot reproduce with 
Eclipse SDK
Version: 2019-12 (4.14)
Build id: I20190916-1800
OS: Linux, v.5.0.0-27-generic, x86_64 / gtk 3.24.8
Java version: 12.0.1
Comment 3 Lars Vogel CLA 2019-09-18 05:19:02 EDT
See Bug 551186 in which the markers are not correctly removed.
Comment 4 Andrey Loskutov CLA 2019-09-18 10:33:50 EDT
(In reply to Lars Vogel from comment #3)
> See Bug 551186 in which the markers are not correctly removed.

Yep, this was most likely part of the problem.

*** This bug has been marked as a duplicate of bug 551186 ***
Comment 5 Lars Vogel CLA 2019-09-18 13:59:22 EDT
Also see this error in one of my workspaces. AFAICS JDT core is writing this. Andrey did you also check changes in JDT? Looks to me that there was a lot of activity 3 days ago.

Example error message:

A cycle was detected in the build path of project 'org.eclipse.core.tools'. The cycle consists of projects {org.eclipse.osgi.services, External Plug-in Libraries, org.eclipse.compare, org.eclipse.ui, org.eclipse.ui.workbench, org.eclipse.e4.core.services, org.eclipse.e4.ui.workbench.swt, org.eclipse.e4.ui.workbench, org.eclipse.e4.ui.model.workbench, org.eclipse.e4.ui.services, org.eclipse.e4.ui.di, org.eclipse.e4.core.di.extensions.supplier, org.eclipse.e4.core.commands, org.eclipse.e4.ui.bindings, org.eclipse.e4.ui.css.swt.theme, org.eclipse.e4.ui.workbench.addons.swt, org.eclipse.e4.ui.workbench.renderers.swt, org.eclipse.ui.ide, org.eclipse.ui.views, org.eclipse.ui.forms, org.eclipse.ui.navigator, org.eclipse.ui.workbench.texteditor, org.eclipse.ui.editors, org.eclipse.compare.examples, org.eclipse.compare.examples.xml, org.eclipse.compare.tests, org.eclipse.core.tests.resources, org.eclipse.team.ui, org.eclipse.core.tests.resources.saveparticipant, org.eclipse.core.tests.resources.saveparticipant1, org.eclipse.core.tests.resources.saveparticipant2, org.eclipse.core.tests.resources.saveparticipant3, org.eclipse.core.tools, org.eclipse.jdt.ui, org.eclipse.search, org.eclipse.ltk.ui.refactoring, org.eclipse.ui.navigator.resources, org.eclipse.ui.views.properties.tabbed, org.eclipse.core.tools.resources, org.eclipse.e4.core.commands.tests, org.eclipse.e4.core.tests, org.eclipse.e4.demo.cssbridge, org.eclipse.e4.emf.xpath.test, org.eclipse.e4.tools, org.eclipse.pde.core, org.eclipse.pde.launching, org.eclipse.e4.tools.bundle.spy, org.eclipse.e4.tools.spy, org.eclipse.e4.tools.compat, org.eclipse.e4.tools.context.spy, org.eclipse.e4.tools.css.spy, org.eclipse.e4.tools.emf.editor3x, org.eclipse.e4.tools.emf.ui, org.eclipse.e4.tools.event.spy, org.eclipse.e4.tools.jdt.templates, org.eclipse.e4.tools.model.spy, org.eclipse.e4.tools.preference.spy, org.eclipse.e4.tools.test, org.eclipse.e4.ui.bindings.tests, org.eclipse.e4.ui.examples.job, org.eclipse.e4.ui.progress, org.eclipse.e4.ui.menu.tests.debug, org.eclipse.e4.ui.menu.tests.p1, org.eclipse.e4.ui.menu.tests.p2, org.eclipse.e4.ui.menu.tests.p3, org.eclipse.e4.ui.menu.tests.p4, org.eclipse.e4.ui.swt.gtk, org.eclipse.e4.ui.tests, org.eclipse.e4.ui.tests.css.swt, org.eclipse.e4.ui.workbench.addons.swt.test, org.eclipse.egit.ui, org.eclipse.equinox.p2.ui, org.eclipse.equinox.p2.ui.sdk, org.eclipse.equinox.p2.ui.sdk.scheduler, org.eclipse.jdt.apt.core, org.eclipse.jdt.apt.pluggable.core, org.eclipse.jdt.apt.pluggable.tests, org.eclipse.jdt.apt.tests, org.eclipse.jdt.apt.ui, org.eclipse.jdt.astview, org.eclipse.jdt.debug.tests, org.eclipse.jdt.debug.ui, org.eclipse.jdt.jeview, org.eclipse.jdt.junit, org.eclipse.jdt.text.tests, org.eclipse.jface.text.tests, org.eclipse.jdt.ui.tests, org.eclipse.jdt.ui.examples.javafamily, org.eclipse.jdt.ui.examples.projects, org.eclipse.jdt.ui.tests.refactoring, org.eclipse.jface.tests, org.eclipse.ui.tests.harness, org.eclipse.jsch.ui, org.eclipse.ltk.ui.refactoring.tests, org.eclipse.pde.ui.templates, org.eclipse.pde.ui.templates.tests, org.eclipse.platform, org.eclipse.search.tests, org.eclipse.team.cvs.ui, org.eclipse.team.examples.filesystem, org.eclipse.team.genericeditor.diff.extension, org.eclipse.team.tests.core, org.eclipse.team.tests.A cycle was detected in the build path of project 'org.eclipse.core.tools'. The cycle consists of projects {org.eclipse.osgi.services, External Plug-in Libraries, org.eclipse.compare, org.eclipse.ui, org.eclipse.ui.workbench, org.eclipse.e4.core.services, org.eclipse.e4.ui.workbench.swt, org.eclipse.e4.ui.workbench, org.eclipse.e4.ui.model.workbench, org.eclipse.e4.ui.services, org.eclipse.e4.ui.di, org.eclipse.e4.core.di.extensions.supplier, org.eclipse.e4.core.commands, org.eclipse.e4.ui.bindings, org.eclipse.e4.ui.css.swt.theme, org.eclipse.e4.ui.workbench.addons.swt, org.eclipse.e4.ui.workbench.renderers.swt, org.eclipse.ui.ide, org.eclipse.ui.views, org.eclipse.ui.forms, org.eclipse.ui.navigator, org.eclipse.ui.workbench.texteditor, org.eclipse.ui.editors, org.eclipse.compare.examples, org.eclipse.compare.examples.xml, org.eclipse.compare.tests, org.eclipse.core.tests.resources, org.eclipse.team.ui, org.eclipse.core.tests.resources.saveparticipant, org.eclipse.core.tests.resources.saveparticipant1, org.eclipse.core.tests.resources.saveparticipant2, org.eclipse.core.tests.resources.saveparticipant3, org.eclipse.core.tools, org.eclipse.jdt.ui, org.eclipse.search, org.eclipse.ltk.ui.refactoring, org.eclipse.ui.navigator.resources, org.eclipse.ui.views.properties.tabbed, org.eclipse.core.tools.resources, org.eclipse.e4.core.commands.tests, org.eclipse.e4.core.tests, org.eclipse.e4.demo.cssbridge, org.eclipse.e4.emf.xpath.test, org.eclipse.e4.tools, org.eclipse.pde.core, org.eclipse.pde.launching, org.eclipse.e4.tools.bundle.spy, org.eclipse.e4.tools.spy, org.eclipse.e4.tools.compat, org.eclipse.e4.tools.context.spy, org.eclipse.e4.tools.css.spy, org.eclipse.e4.tools.emf.editor3x, org.eclipse.e4.tools.emf.ui, org.eclipse.e4.tools.event.spy, org.eclipse.e4.tools.jdt.templates, org.eclipse.e4.tools.model.spy, org.eclipse.e4.tools.preference.spy, org.eclipse.e4.tools.test, org.eclipse.e4.ui.bindings.tests, org.eclipse.e4.ui.examples.job, org.eclipse.e4.ui.progress, org.eclipse.e4.ui.menu.tests.debug, org.eclipse.e4.ui.menu.tests.p1, org.eclipse.e4.ui.menu.tests.p2, org.eclipse.e4.ui.menu.tests.p3, org.eclipse.e4.ui.menu.tests.p4, org.eclipse.e4.ui.swt.gtk, org.eclipse.e4.ui.tests, org.eclipse.e4.ui.tests.css.swt, org.eclipse.e4.ui.workbench.addons.swt.test, org.eclipse.egit.ui, org.eclipse.equinox.p2.ui, org.eclipse.equinox.p2.ui.sdk, org.eclipse.equinox.p2.ui.sdk.scheduler, org.eclipse.jdt.apt.core, org.eclipse.jdt.apt.pluggable.core, org.eclipse.jdt.apt.pluggable.tests, org.eclipse.jdt.apt.tests, org.eclipse.jdt.apt.ui, org.eclipse.jdt.astview, org.eclipse.jdt.debug.tests, org.eclipse.jdt.debug.ui, org.eclipse.jdt.jeview, org.eclipse.jdt.junit, org.eclipse.jdt.text.tests, org.eclipse.jface.text.tests, org.eclipse.jdt.ui.tests, org.eclipse.jdt.ui.examples.javafamily, org.eclipse.jdt.ui.examples.projects, org.eclipse.jdt.ui.tests.refactoring, org.eclipse.jface.tests, org.eclipse.ui.tests.harness, org.eclipse.jsch.ui, org.eclipse.ltk.ui.refactoring.tests, org.eclipse.pde.ui.templates, org.eclipse.pde.ui.templates.tests, org.eclipse.platform, org.eclipse.search.tests, org.eclipse.team.cvs.ui, org.eclipse.team.examples.filesystem, org.eclipse.team.genericeditor.diff.extension, org.eclipse.team.tests.core, org.eclipse.team.tests.cvs.core, org.eclipse.test, org.eclipse.ui.ide.application, org.eclipse.text.quicksearch, org.eclipse.text.quicksearch.tests, org.eclipse.tips.examples, org.eclipse.tips.ide, org.eclipse.tips.tests, org.eclipse.ui.browser, org.eclipse.ui.editors.tests, org.eclipse.ui.examples.adapterservice, org.eclipse.ui.examples.contributions, org.eclipse.ui.examples.fieldassist, org.eclipse.ui.examples.javaeditor, org.eclipse.ui.examples.job, org.eclipse.ui.examples.multipageeditor, org.eclipse.ui.examples.navigator, org.eclipse.ui.examples.propertysheet, org.eclipse.ui.examples.readmetool, org.eclipse.ui.examples.undo, org.eclipse.ui.examples.uriSchemeHandler, org.eclipse.ui.examples.views.properties.tabbed.article, org.eclipse.ui.forms.examples, org.eclipse.ui.genericeditor, org.eclipse.ui.genericeditor.examples, org.eclipse.ui.genericeditor.tests, org.eclipse.ui.workbench.texteditor.tests, org.eclipse.ui.ide.application.tests, org.eclipse.ui.monitoring, org.eclipse.ui.monitoring.tests, org.eclipse.ui.net, org.eclipse.ui.tests, org.eclipse.ui.tests.browser, org.eclipse.ui.tests.forms, org.eclipse.ui.tests.navigator, org.eclipse.ui.tests.performance, org.eclipse.ui.tests.pluginchecks, org.eclipse.ui.tests.rcp, org.eclipse.ui.tests.views.properties.tabbed, org.eclipse.ui.views.log}cvs.core, org.eclipse.test, org.eclipse.ui.ide.application, org.eclipse.text.quicksearch, org.eclipse.text.quicksearch.tests, org.eclipse.tips.examples, org.eclipse.tips.ide, org.eclipse.tips.tests, org.eclipse.ui.browser, org.eclipse.ui.editors.tests, org.eclipse.ui.examples.adapterservice, org.eclipse.ui.examples.contributions, org.eclipse.ui.examples.fieldassist, org.eclipse.ui.examples.javaeditor, org.eclipse.ui.examples.job, org.eclipse.ui.examples.multipageeditor, org.eclipse.ui.examples.navigator, org.eclipse.ui.examples.propertysheet, org.eclipse.ui.examples.readmetool, org.eclipse.ui.examples.undo, org.eclipse.ui.examples.uriSchemeHandler, org.eclipse.ui.examples.views.properties.tabbed.article, org.eclipse.ui.forms.examples, org.eclipse.ui.genericeditor, org.eclipse.ui.genericeditor.examples, org.eclipse.ui.genericeditor.tests, org.eclipse.ui.workbench.texteditor.tests, org.eclipse.ui.ide.application.tests, org.eclipse.ui.monitoring, org.eclipse.ui.monitoring.tests, org.eclipse.ui.net, org.eclipse.ui.tests, org.eclipse.ui.tests.browser, org.eclipse.ui.tests.forms, org.eclipse.ui.tests.navigator, org.eclipse.ui.tests.performance, org.eclipse.ui.tests.pluginchecks, org.eclipse.ui.tests.rcp, org.eclipse.ui.tests.views.properties.tabbed, org.eclipse.ui.views.log}

If the above is an ordered list, than it is already wrong in the first entry, AFAICS 'org.eclipse.core.tools' has no dependency to org.eclipse.osgi.services.
Comment 6 Lars Vogel CLA 2019-09-18 14:06:48 EDT
Created attachment 279918 [details]
PDE dependencies

The PDE dependencies in Eclipse SDK
Version: 2019-09 (4.13)
Build id: I20190826-0415
OS: Linux, v.5.0.0-27-generic, x86_64 / gtk 3.24.8, WebKit 2.24.4
Java version: 12.0.1
Comment 7 Lars Vogel CLA 2019-09-18 14:14:06 EDT
Created attachment 279919 [details]
PDE dependencies new

PDE dependencies in

Eclipse SDK
Version: 2019-12 (4.14)
Build id: I20190916-1800
OS: Linux, v.5.0.0-27-generic, x86_64 / gtk 3.24.8
Java version: 12.0.1

Looks like the dependencies are the same but they are differently sorted (why?)
Comment 8 Lars Vogel CLA 2019-09-18 15:29:22 EDT
Will revert Bug 550038 to exclude it as possible reason.
Comment 9 Lars Vogel CLA 2019-09-18 16:20:25 EDT
(In reply to Lars Vogel from comment #8)
> Will revert Bug 550038 to exclude it as possible reason.

Done.
Comment 10 Andrey Loskutov CLA 2019-09-19 02:57:09 EDT
(In reply to Lars Vogel from comment #8)
> Will revert Bug 550038 to exclude it as possible reason.

Didn't help.

Today started same workspace with the new build I20190918-1800, updated everything to master latest state, was all OK (except bug 551109 error is still there and my beloved SWT API errors bug 532282).

Restart => ALL ~150 projects show "A cycle was detected in the build path of project" errors, same as in comment 0.

This must be a regression in 4.14 or late 4.13 RC builds (where I was on vacation).

To get rid of cycle errors I have to close and reopen all ~150 projects (hint from bug 551186), but after endless rebuilds I see *few* cycle errors appearing again... After another one restart, again almost ALL ~150 projects show cycle errors. What?!?

Also it is really strange that even such "basic" bundles like org.eclipse.equinox.preferences show cycle errors. I assume the (simple) logic detects cycle in one of the *users* of such bundles but attaches the error to ALL *dependencies* of the bundle where the cycle was found.

A cycle was detected in the build path of project 'org.eclipse.equinox.preferences'. The cycle consists of projects {org.eclipse.equinox.preferences, org.eclipse.compare, org.eclipse.ui, org.eclipse.core.runtime, org.eclipse.core.contenttype, org.eclipse.ui.workbench, org.eclipse.core.expressions, org.eclipse.jface.text, org.eclipse.text, org.eclipse.e4.core.services, org.eclipse.e4.ui.workbench.swt, org.eclipse.e4.ui.workbench, org.eclipse.e4.ui.model.workbench, org.eclipse.e4.ui.services, org.eclipse.e4.ui.di, org.eclipse.e4.core.di.extensions.supplier, org.eclipse.e4.core.commands, org.eclipse.e4.ui.dialogs, org.eclipse.e4.ui.css.core, org.eclipse.e4.ui.css.swt, org.eclipse.e4.ui.bindings, org.eclipse.e4.ui.workbench3, org.eclipse.e4.ui.css.swt.theme, org.eclipse.e4.ui.workbench.addons.swt, org.eclipse.e4.ui.workbench.renderers.swt, org.eclipse.core.resources, org.eclipse.ui.ide, org.eclipse.ui.views, org.eclipse.ui.forms, org.eclipse.ui.navigator, org.eclipse.ui.workbench.texteditor, org.eclipse.compare.core, org.eclipse.ui.editors, org.eclipse.core.filebuffers, org.eclipse.core.externaltools, org.eclipse.debug.core, org.eclipse.core.variables, org.eclipse.core.net, org.eclipse.core.resources.spysupport, org.eclipse.core.tools.resources, org.eclipse.debug.ui, org.eclipse.ui.console, org.eclipse.ui.genericeditor, org.eclipse.e4.ui.progress, org.eclipse.e4.ui.swt.gtk, org.eclipse.jdt.apt.core, org.eclipse.jdt.core, org.eclipse.team.core, org.eclipse.jdt.compiler.apt, org.eclipse.jdt.compiler.tool, org.eclipse.jdt.apt.pluggable.core, org.eclipse.jdt.astview, org.eclipse.jdt.ui, org.eclipse.search, org.eclipse.ltk.core.refactoring, org.eclipse.ltk.ui.refactoring, org.eclipse.team.ui, org.eclipse.jdt.launching, org.eclipse.jdt.debug, org.eclipse.ui.navigator.resources, org.eclipse.ui.views.properties.tabbed, org.eclipse.jdt.core.manipulation, org.eclipse.jdt.debug.ui, org.eclipse.jdt.junit, org.eclipse.jdt.junit.core, org.eclipse.jsch.core, org.eclipse.jsch.ui, org.eclipse.pde.api.tools, org.eclipse.pde.core, org.eclipse.pde.build, org.eclipse.pde.api.tools.generator, org.eclipse.pde.api.tools.ui, org.eclipse.pde.ui, org.eclipse.ui.views.log, org.eclipse.pde.launching, org.eclipse.ui.trace, org.eclipse.pde.ds.annotations, org.eclipse.pde.ds.core, org.eclipse.pde.ds.ui, org.eclipse.pde.genericeditor.extension, org.eclipse.pde.junit.runtime, org.eclipse.pde.runtime, org.eclipse.pde.ua.core, org.eclipse.pde.ua.ui, org.eclipse.pde.ui.templates, org.eclipse.swt.examples.browser.demos, org.eclipse.swt.examples.launcher, org.eclipse.swt.examples.views, org.eclipse.team.genericeditor.diff.extension, org.eclipse.text.quicksearch, org.eclipse.tools.layout.spy, org.eclipse.ui.browser, org.eclipse.ui.externaltools, org.eclipse.ui.ide.application, org.eclipse.urischeme, org.eclipse.ui.monitoring, org.eclipse.ui.net}

Looking on bug 550738 change I wonder if we see a regression from job cancellation fix - assuming we re-scheduled the build before that is now does not happen? But this would imply that the manual clean build that I execute triggers some jobs that get cancelled and was *reliably* not cancelled all the time before?

But beside this, I simply do not see a good candidate for the regression root cause (PDE/JDT/runtime etc) in git history.
Comment 11 Andrey Loskutov CLA 2019-09-19 03:06:06 EDT
In the related code around DeltaProcessor.java I see changes from bug 466299.

@Stephan: could be that we have now some new cycles appearing due external annotation paths?
Comment 12 Lars Vogel CLA 2019-09-19 08:19:31 EDT
Added a tracing option for cycles, see Bug 551237
Comment 13 Stephan Herrmann CLA 2019-09-19 09:16:22 EDT
(In reply to Andrey Loskutov from comment #11)
> In the related code around DeltaProcessor.java I see changes from bug 466299.
> 
> @Stephan: could be that we have now some new cycles appearing due external
> annotation paths?

How many projects in your workspace use external annotations, and find those external annotations in a project that would otherwise not be visible to the current project?

As you ask about DeltaProcessor: that guy only *remembers* those extra attributes that are used for things like external annotation path, module tweaks etc.pp (no *evaluation* at that point). I don't see how this change could contribute to project cycles.
Comment 14 Andrey Loskutov CLA 2019-09-19 09:21:19 EDT
(In reply to Stephan Herrmann from comment #13)
> (In reply to Andrey Loskutov from comment #11)
> > In the related code around DeltaProcessor.java I see changes from bug 466299.
> > 
> > @Stephan: could be that we have now some new cycles appearing due external
> > annotation paths?
> 
> How many projects in your workspace use external annotations, and find those
> external annotations in a project that would otherwise not be visible to the
> current project?

I have mostly SDK projects. If you have a hint how I can grep for the external annotations in config files, I could answer this :-)
Comment 15 Stephan Herrmann CLA 2019-09-19 09:25:03 EDT
(In reply to Andrey Loskutov from comment #14)
> (In reply to Stephan Herrmann from comment #13)
> > (In reply to Andrey Loskutov from comment #11)
> > > In the related code around DeltaProcessor.java I see changes from bug 466299.
> > > 
> > > @Stephan: could be that we have now some new cycles appearing due external
> > > annotation paths?
> > 
> > How many projects in your workspace use external annotations, and find those
> > external annotations in a project that would otherwise not be visible to the
> > current project?
> 
> I have mostly SDK projects. If you have a hint how I can grep for the
> external annotations in config files, I could answer this :-)

Search for "annotationpath" in .classpath
Comment 16 Andrey Loskutov CLA 2019-09-19 09:54:36 EDT
(In reply to Stephan Herrmann from comment #15)
> > I have mostly SDK projects. If you have a hint how I can grep for the
> > external annotations in config files, I could answer this :-)
> 
> Search for "annotationpath" in .classpath

Thanks. Zero matches :-(
Comment 17 Lars Vogel CLA 2019-09-19 10:17:59 EDT
(In reply to Andrey Loskutov from comment #16)
> Thanks. Zero matches :-(

IIRC PDE adds DS annotations JAR dynamically if you have the Plug-in Development -> DS Annotations -> Generate... preference active
Comment 18 Lars Vogel CLA 2019-09-19 13:55:50 EDT
(In reply to Andrey Loskutov from comment #11)
> In the related code around DeltaProcessor.java I see changes from bug 466299.

If I revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 from Bug 466299 and do a local SDK build the cycle errors are gone. The official build still shows the cycles my "all Eclipse top-level projects" workspace.
Comment 19 Stephan Herrmann CLA 2019-09-19 14:30:07 EDT
(In reply to Lars Vogel from comment #18)
> (In reply to Andrey Loskutov from comment #11)
> > In the related code around DeltaProcessor.java I see changes from bug 466299.
> 
> If I revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 from Bug 466299 and do
> a local SDK build the cycle errors are gone. The official build still shows
> the cycles my "all Eclipse top-level projects" workspace.

I'm currently installing according to https://wiki.eclipse.org/Eclipse_Platform_SDK_Provisioning to have a look.
Comment 20 Stephan Herrmann CLA 2019-09-19 16:10:28 EDT
(In reply to Stephan Herrmann from comment #19)
> (In reply to Lars Vogel from comment #18)
> > (In reply to Andrey Loskutov from comment #11)
> > > In the related code around DeltaProcessor.java I see changes from bug 466299.
> > 
> > If I revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 from Bug 466299 and do
> > a local SDK build the cycle errors are gone. The official build still shows
> > the cycles my "all Eclipse top-level projects" workspace.
> 
> I'm currently installing according to
> https://wiki.eclipse.org/Eclipse_Platform_SDK_Provisioning to have a look.

Just now the freshly provisioned workspace completed building with no errors.

Restart => Still OK

Clean build of org.eclipse.equinox.preferences (see comment 10) => Still OK
Comment 21 Stephan Herrmann CLA 2019-09-19 16:16:12 EDT
(In reply to Lars Vogel from comment #5)
> A cycle was detected in the build path of project 'org.eclipse.core.tools'.
> The cycle consists of projects {org.eclipse.osgi.services, External Plug-in
> Libraries, org.eclipse.compare, org.eclipse.ui, org.eclipse.ui.workbench,
> org.eclipse.e4.core.services, org.eclipse.e4.ui.workbench.swt,
> [...]
> org.eclipse.ui.tests.pluginchecks, org.eclipse.ui.tests.rcp,
> org.eclipse.ui.tests.views.properties.tabbed, org.eclipse.ui.views.log}
> 
> If the above is an ordered list, than it is already wrong in the first
> entry, AFAICS 'org.eclipse.core.tools' has no dependency to
> org.eclipse.osgi.services.

I checked the code generating this message: yes, the list is ordered.

But mind you that the cycle can start at an indirect dependency of the focus project: a -> b -> c -> d -> e -> c will report the cycle {c, d, e}

I don't have org.eclipse.core.tools, so I could not test building that project.
Comment 22 Andrey Loskutov CLA 2019-09-19 16:18:13 EDT
Stephan, I saw now this cycle errors after updating one nightly build with the next one. By "update" I mean replacement of ony binaries with others, preserving same workspace. Download SDK, extract in same location (old one removed before), install egit etc tooling by usimg a temporary workspace, switch to old workspace. The routine I do almost every day. Two times in the row now I see this cycle errors, starting (after my vacation) with build I20190916-1800. The errors seem to disappear after some voodoo. I never saw this behavior before.
Comment 23 Stephan Herrmann CLA 2019-09-19 17:06:46 EDT
(In reply to Andrey Loskutov from comment #22)
> Stephan, I saw now this cycle errors after updating one nightly build with
> the next one. By "update" I mean replacement of ony binaries with others,
> preserving same workspace. Download SDK, extract in same location (old one
> removed before), install egit etc tooling by usimg a temporary workspace,
> switch to old workspace. The routine I do almost every day. Two times in the
> row now I see this cycle errors, starting (after my vacation) with build
> I20190916-1800. The errors seem to disappear after some voodoo. I never saw
> this behavior before.

I already suspected s.t. in persistent build state to have a finger in the pie, so I opened the same workspace first with R4.13 then with the latest I-build. S.t. went wrong with dependency resolution in R4.13, so I had tens of thousands of compile errors, but no build path error. But then I was mostly interested in going back from R4.13 to latest I-build but here building actually resolved most errors (except for a remainder of 118 errors - again no build path / cycle errors).

Following your recommendation I fetched an I-build SDK before the change in Bug 466299, now the workspace has 127244 compile errors. But again the interesting part of the experiment is going back to latest I-build.

And: Bingo: It stopped at my breakpoint during cycle detection ...
Comment 24 Stephan Herrmann CLA 2019-09-19 17:18:54 EDT
org.eclipse.osgi.services has this resolved classpath entry:

/org.eclipse.osgi.services[CPE_PROJECT][K_SOURCE][isExported:false][AccessRuleSet {pattern=org/osgi/service/component/annotations/* (ACCESSIBLE), pattern=**/* (NON ACCESSIBLE | IGNORE IF BETTER)} [classpath entry: org.eclipse.osgi.services]]
[combine access rules:true]


At first we only record this one-element cycle, but I already have an idea why the error message may become bloated with more projects.

But first of all: who creates that resolved CPE?
Comment 25 Stephan Herrmann CLA 2019-09-19 17:35:38 EDT
In org.eclipse.pde.internal.core.RequiredPluginsInitializer.initialize(IPath, IJavaProject) I see this:

project = P/org.eclipse.osgi.services

model ("org.eclipse.osgi.services")
 - fBundleDescription (org.eclipse.osgi.services_3.8.0.qualifier) (id=2465)
   - dependencies
     [1] (org.eclipse.osgi.services_3.8.0.qualifier) (id=2514)
         - dependencies
           [1] (org.eclipse.osgi.services_3.8.0.qualifier) (id=2465)

Note, that we have two distinct instances of BundleDescriptionImpl both representing org.eclipse.osgi.services_3.8.0.qualifier, but then there's a cycle already at this level.

Over to PDE.
Comment 26 Andrey Loskutov CLA 2019-09-19 17:46:26 EDT
(In reply to Stephan Herrmann from comment #24)
> I already have an idea
> why the error message may become bloated with more projects.

Could you share this idea?
Comment 27 Stephan Herrmann CLA 2019-09-19 17:52:56 EDT
(In reply to Stephan Herrmann from comment #24)
> At first we only record this one-element cycle, but I already have an idea
> why the error message may become bloated with more projects.

=> bug 551284
Comment 28 Lars Vogel CLA 2019-09-20 03:25:04 EDT
(In reply to Lars Vogel from comment #18)
> (In reply to Andrey Loskutov from comment #11)
> > In the related code around DeltaProcessor.java I see changes from bug 466299.
> 
> If I revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 from Bug 466299 and do
> a local SDK build the cycle errors are gone. The official build still shows
> the cycles my "all Eclipse top-level projects" workspace.

Andrey, can you confirm that reverting f3a7aa49834eef8027d0f4162b6d5450cda0baa2 fixes the issue for you?
Comment 29 Andrey Loskutov CLA 2019-09-20 03:34:07 EDT
(In reply to Lars Vogel from comment #28)
> Andrey, can you confirm that reverting
> f3a7aa49834eef8027d0f4162b6d5450cda0baa2 fixes the issue for you?

No, I haven't tested this.
Comment 30 Lars Vogel CLA 2019-09-20 03:36:20 EDT
(In reply to Andrey Loskutov from comment #29)
> (In reply to Lars Vogel from comment #28)
> > Andrey, can you confirm that reverting
> > f3a7aa49834eef8027d0f4162b6d5450cda0baa2 fixes the issue for you?
> 
> No, I haven't tested this.

Please do
Comment 31 Stephan Herrmann CLA 2019-09-20 10:05:46 EDT
(In reply to Lars Vogel from comment #28)
> (In reply to Lars Vogel from comment #18)
> > (In reply to Andrey Loskutov from comment #11)
> > > In the related code around DeltaProcessor.java I see changes from bug 466299.
> > 
> > If I revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 from Bug 466299 and do
> > a local SDK build the cycle errors are gone. The official build still shows
> > the cycles my "all Eclipse top-level projects" workspace.
> 
> Andrey, can you confirm that reverting
> f3a7aa49834eef8027d0f4162b6d5450cda0baa2 fixes the issue for you?

From extensive debugging the observed root cause is in PDE creating a wrong model for org.eclipse.osgi.services. If that project depends on itself that surely is not the intended semantics, is it?

I would be much surprised when this cause in PDE is in turn *caused* by a change in JDT. Does PDE need information from JDT in order to interpret MANIFEST.MF??

Until we have evidence of the influence from JDT I'd rather assume that the error is in PDE and that it depends on some unknown, non-deterministic factor. 

In fact, if the change in JDT makes the bug in PDE appear more frequently, I would suggest to use the opportunity, rather than making the bug more rare again.
Comment 32 Andrey Loskutov CLA 2019-09-20 10:20:07 EDT
Stephan, do you think bug 551186 is in PDE or JDT? I'm currently unable to use the IDE because I have to open/close all the projects to get rid of the cycle errors for a few minutes :-(
Comment 33 Lars Vogel CLA 2019-09-20 10:25:26 EDT
As this is a major regression (SDK not usable to develop) it is required to get fixed or the change that causes it must be reverted. AFAICS reverting the JDT change fixes it. I don't see a change in PDE which could cause the regression.
Comment 34 Andrey Loskutov CLA 2019-09-20 10:32:15 EDT
(In reply to Lars Vogel from comment #33)
> As this is a major regression (SDK not usable to develop) it is required to
> get fixed or the change that causes it must be reverted. AFAICS reverting
> the JDT change fixes it. I don't see a change in PDE which could cause the
> regression.

I agree. I'm unable to use recent builds.

> > If I revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 from Bug 466299 and do
> > a local SDK build the cycle errors are gone.

Lars, are you 100% sure that this "fixes" the problem? If yes, I would vote for reverting. Stephan: do you experience cycle errors in recent builds?
Comment 35 Andrey Loskutov CLA 2019-09-20 10:35:52 EDT
I would say: let us revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 and have one SDK build without this commit to see if this is really the root cause. It is easier to do now, as later we might have merge issues etc, the patch is not a smallest one.

Stephan: WDYT?
Comment 36 Lars Vogel CLA 2019-09-20 10:39:39 EDT
Andrey, I'm not 100% sure that is why I ask you to verify.
Comment 37 Stephan Herrmann CLA 2019-09-20 10:59:37 EDT
(In reply to Andrey Loskutov from comment #35)
> I would say: let us revert f3a7aa49834eef8027d0f4162b6d5450cda0baa2 and have
> one SDK build without this commit to see if this is really the root cause.
> It is easier to do now, as later we might have merge issues etc, the patch
> is not a smallest one.
> 
> Stephan: WDYT?

I debugged it to a point where PDE creates a wrong model. Shouldn't s.o. debug PDE, why it misbehaves?

But if you MUST, go ahead and revert AND MAKE SURE you keep a "broken" build for further debugging, because the issue is not resolved before we understand the root cause. Reverting cannot possibly achieve this.

BTW, I am aware that the JDT change has some risk, that's why I pushed it very early in the cycle :)

(In reply to Andrey Loskutov from comment #32)
> Stephan, do you think bug 551186 is in PDE or JDT? I'm currently unable to
> use the IDE because I have to open/close all the projects to get rid of the
> cycle errors for a few minutes :-(

I don't know and can't analyse right now. Have you tried simple deleting the error markers?
Comment 38 Andrey Loskutov CLA 2019-09-20 11:04:07 EDT
(In reply to Lars Vogel from comment #36)
> Andrey, I'm not 100% sure that is why I ask you to verify.

OK, I've triggered the build with revert locally, will be done in the next hour. 
Question: did you understand what are the reproducible steps needed to:
 
1) get cycles?
2) get rid of them without closing all projects?

For me this is a mistery. So far the errors seem to appear on every second start, but I can't understand what causes them.
Comment 39 Lars Vogel CLA 2019-09-20 11:11:17 EDT
In my setup the amount of errors written my JDT made the IDE unusable (UI freezes all the time) so I could not do a lot of testing. In my local build with the revert everything works again.
Comment 40 Andrey Loskutov CLA 2019-09-20 11:27:46 EDT
(In reply to Lars Vogel from comment #39)
> In my setup the amount of errors written my JDT made the IDE unusable (UI
> freezes all the time) so I could not do a lot of testing. In my local build
> with the revert everything works again.

In my setup (I'm on I20190918-1800 now) I see following:

1) Once the cycle error appear, I have to close ALL projects to get rid of errors.
2) After that working in the same session with some opened projects is possible.
3) Once IDE is restarted, some projects start to show "Access restriction" errors like "Access restriction: The method 'Preferences.get(String, String)' is not API (restriction on required project 'org.eclipse.equinox.preferences')" - note, that I *have* org.eclipse.equinox.preferences in my workspace, and that I also never seen such kind of errors before.
4) Those errors go away only by clean build of all projects.
5) Some restarts later I see cycle errors. However, I have no idea what exact causes cycles to appear again.
Comment 41 Andrey Loskutov CLA 2019-09-20 12:16:16 EDT
OK, I have tried a custom build without the commit f3a7aa49834eef8027d0f4162b6d5450cda0baa2, based on latest I20190919-1800.

It looks initially OK (I assume the PDE errors seem to be "fixed" for the first time after startup), but after few "good" restarts I see again cycle errors.

This time they seem to disappear after clean build of all projects + restart, but it looks like the commit is not related.
Comment 42 Stephan Herrmann CLA 2019-09-20 14:53:11 EDT
(In reply to Stephan Herrmann from comment #25)
> In
> org.eclipse.pde.internal.core.RequiredPluginsInitializer.initialize(IPath,
> IJavaProject) I see this:
> 
> project = P/org.eclipse.osgi.services
> 
> model ("org.eclipse.osgi.services")
>  - fBundleDescription (org.eclipse.osgi.services_3.8.0.qualifier) (id=2465)
>    - dependencies
>      [1] (org.eclipse.osgi.services_3.8.0.qualifier) (id=2514)
>          - dependencies
>            [1] (org.eclipse.osgi.services_3.8.0.qualifier) (id=2465)
> 
> Note, that we have two distinct instances of BundleDescriptionImpl both
> representing org.eclipse.osgi.services_3.8.0.qualifier, but then there's a
> cycle already at this level.
> 
> Over to PDE.

No longer sure if PDE or Equinox is at the core of it:

* org.eclipse.osgi.services imports several of its own exported packages
  (looks like vodoo but doesn't seem to illegal per se)

* BundleDescriptionImpl.addDependency() rejects to add a self cycle by checking
	if (bundle == this)
		return;

* during the debug session of comment 25 this check was ineffective because
  we had two instances of BundleDescriptionImpl representing the same 
  workspace bundle.

So we are looking for the code that creates duplicate BundleDescriptionImpl, possibly from persisted state?
Comment 43 Andrey Loskutov CLA 2019-09-20 15:09:01 EDT
Yes, it looks like PDE "knows" two versions of the same bundle. I've tried to use SDK to debug Eclipse (at the moment I've managed to get rid of cycles), and launch validation dialog popped up with almost all singleton bundles reported as being duplicated.

Also interesting that not all bundles may have cycle error and I' ve also seen single jdt apt bundle with cycle that contained only that one bundle.

This all looks like a race condition at startup, or something basic broken like object hashcodes or the likes.
Comment 44 Stephan Herrmann CLA 2019-09-20 15:29:23 EDT
After observing the first round to remain cycle-free, I see a second instance created in this stack trace:

BundleDescriptionImpl(BaseDescriptionImpl).setName(String) line: 47	
BundleDescriptionImpl.setSymbolicName(String) line: 282	
StateBuilder.createBundleDescription(StateImpl, Dictionary<String,String>, String) line: 97	
StateObjectFactoryImpl.createBundleDescription(State, Dictionary<String,String>, String, long) line: 35	
PDEState(MinimalState).addBundle(Map<String,String>, File, long) line: 102	
PDEState(MinimalState).addBundle(File, long) line: 125	
PDEState(MinimalState).addBundle(IPluginModelBase, boolean) line: 87	
PluginModelManager.addWorkspaceBundleToState(Map<String,LocalModelEntry>, IPluginModelBase) line: 759	
PluginModelManager.addWorkspaceBundleToState(IPluginModelBase) line: 735	
PluginModelManager.handleAdd(String, IPluginModelBase, PluginModelDelta) line: 880	
PluginModelManager.modelsChanged(IModelProviderEvent) line: 261	
WorkspacePluginModelManager(AbstractModelManager).fireModelProviderEvent(IModelProviderEvent) line: 37	
WorkspacePluginModelManager(WorkspaceModelManager).createAndFireEvent(String, int, Collection<IModel>, Collection<IModel>, Collection<IModel>) line: 279	
WorkspacePluginModelManager.createAndFireEvent(String, int, Collection<IModel>, Collection<IModel>, Collection<IModel>) line: 576	
WorkspacePluginModelManager(WorkspaceModelManager).processModelChanges(String, ArrayList<ModelChange>) line: 258	
WorkspacePluginModelManager(WorkspaceModelManager).processModelChanges() line: 216	
WorkspacePluginModelManager.processModelChanges() line: 565	
WorkspacePluginModelManager(WorkspaceModelManager).resourceChanged(IResourceChangeEvent) line: 134	
DeltaProcessingState$1.run() line: 472	
SafeRunner.run(ISafeRunnable) line: 45	
DeltaProcessingState.resourceChanged(IResourceChangeEvent) line: 465	
NotificationManager$1.run() line: 305	
SafeRunner.run(ISafeRunnable) line: 45	
NotificationManager.notify(ResourceChangeListenerList$ListenerEntry[], ResourceChangeEvent, boolean) line: 295	
NotificationManager.broadcastChanges(ElementTree, ResourceChangeEvent, boolean) line: 158	
Workspace.broadcastPostChange() line: 379	
Workspace.endOperation(ISchedulingRule, boolean) line: 1498	
ExternalFoldersManager$1InitializeFolders(InternalWorkspaceJob).run(IProgressMonitor) line: 49	
Worker.run() line: 63	


At org.eclipse.pde.internal.core.PluginModelManager.addWorkspaceBundleToState(Map<String, LocalModelEntry>, IPluginModelBase) I see an entry (PluginModelManager$LocalModelEntry) having two elements in fWorkspaceEntries: the instance from initial resolving plus a new one from the change event (each one wrapped in its own BundlePluginModel). 
At this particular location it is not a hashCode/equals issue as fWorkspaceEntries is an ArrayList.

Here Equinox structures and PDE show the same duplication.
Comment 45 Andrey Loskutov CLA 2019-09-20 18:11:21 EDT
Thomas, this brings me to the idea that changes for bug 550645 could be the root cause: https://git.eclipse.org/r/#/c/148986/ and / or https://git.eclipse.org/r/#/c/149272/.

I've tried to understand the new implementation of put, but failed:

https://git.eclipse.org/r/#/c/148986/8/bundles/org.eclipse.osgi/container/src/org/eclipse/osgi/framework/util/CaseInsensitiveDictionaryMap.java@191

Could it be that we might have now data races due new "put" implementation? This would explain both the randomness of the issue and the duplication of the data.
Comment 46 Andrey Loskutov CLA 2019-09-21 15:36:38 EDT
OK, I have one workspace that seem to show this errors reliably after closing projects below, restart, opening projects, restart -> cycles.

org.eclipse.osgi
org.eclipse.osgi.compatibility.state
org.eclipse.osgi.services

If I start the workspace with build I20190909-1520, ALL cycle errors disappear and everything compiles again. I can restart/open/close projects without any problem. But it is enough to start the same workspace with I20190914-1800 build to get cycle errors immediately.

So the regression is definitely in the range (I20190909-1520, I20190914-1800], unfortunately exact the time after my vacation where I was not using Eclipse nightly builds.

Still trying to isolate the example workspace needed.
Right now I'm down to those bundles in the workspace:

org.eclipse.e4.ui.bindings
org.eclipse.equinox.common
org.eclipse.equinox.registry
org.eclipse.jdt.apt.core
org.eclipse.jdt.apt.ui
* org.eclipse.osgi
* org.eclipse.osgi.compatibility.state
* org.eclipse.osgi.services
org.eclipse.pde.api.tools
org.eclipse.pde.api.tools.annotations
org.eclipse.pde.api.tools.ui
org.eclipse.pde.core
org.eclipse.pde.runtime
org.eclipse.pde.ui
Comment 47 Andrey Loskutov CLA 2019-09-22 03:50:55 EDT
Created attachment 279970 [details]
duplicated bundles

The minimal example so far is:

org.eclipse.osgi
org.eclipse.osgi.compatibility.state
org.eclipse.osgi.services
org.eclipse.pde.core
org.eclipse.pde.runtime
org.eclipse.pde.ui

One thing: it looks like a java editor (in my case from org.eclipse.pde.core) must be opened (that triggers classpath container init on the main thread) to increase the probability of cycle errors. Also "PDE -> include all plugins from target in Java workspace scope" must be enabled

The steps are also (high probability to succeed):
1) Import projects above into the workspace, open MinimalState.java editor and keep it open all the next steps.
2) Restart
3) Close 3 osgi projects (org.eclipse.pde.core will get compilation errors but not cycle errors), restart => no errors.
4) Open 3 osgi projects (org.eclipse.pde.core will be still error free), restart
5) At this step there is a high possibility to see cycle errors on org.eclipse.osgi.services, org.eclipse.pde.runtime and org.eclipse.pde.ui. If not, open/close org.eclipse.pde.runtime and org.eclipse.pde.ui projects + restart few times.

Funny enough, in this minimal example, cycle errors disappear on next or next few consecutive restarts *without doing anything else* (on my main workspace this is not the case), and appear again after next few restarts without any user interaction with IDE.

Also interesting, that I see JDT indexer running on every start, even if there were no changes between restarts. On a "good" SDK build this happens only after closing or opening projects. I assume this happens due the constant duplication/deduplication of found PDE dependencies on startup.
Comment 48 Andrey Loskutov CLA 2019-09-22 16:02:34 EDT
It is a regression in platform resources, commit http://git.eclipse.org/c/platform/eclipse.platform.resources.git/commit/?id=c1939c1f11f751d626f0ce870d3d4e39eeb81a79 for bug 540953 is it.

See
https://git.eclipse.org/r/#/c/147600/7/bundles/org.eclipse.core.resources/src/org/eclipse/core/internal/resources/Workspace.java@2199

Inside of many innocent monitor optimizations a call to the notificationManager.startup(null); was *removed* (I assume because it is called before in startup() too). The extra comment that explained *why* the call was needed, was also not understood and moved to the wrong place.

I've totally ignored this commit while scanning platform repos for the possible root cause because neither the commit summary nor message says anything about the "optimization" and removed notificationManager.startup() call.

Another good example that one should never mix many unrelated changes in one commit, and at same time also another good example that one should not "optimize" code that one doesn't know and doesn't really understand.

Patch follows.
Comment 49 Eclipse Genie CLA 2019-09-22 16:04:26 EDT
New Gerrit change created: https://git.eclipse.org/r/149956
Comment 51 Lars Vogel CLA 2019-09-23 00:37:22 EDT
Thanks, Andrey for fixing.
Comment 52 Andrey Loskutov CLA 2019-09-23 04:34:51 EDT
Works fine with I20190922-1800.