Bug 578404 - Occasional freeze of ApiToolsPluginTestSuite
Summary: Occasional freeze of ApiToolsPluginTestSuite
Status: NEW
Alias: None
Product: PDE
Classification: Eclipse Project
Component: UI (show other bugs)
Version: 4.23   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: PDE-UI-Inbox CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords:
Depends on:
Blocks:
 
Reported: 2022-01-27 03:41 EST by Hannes Wellmann CLA
Modified: 2024-03-01 00:27 EST (History)
2 users (show)

See Also:


Attachments
freeze 1 (15.55 KB, text/x-log)
2022-01-27 03:49 EST, Andrey Loskutov CLA
no flags Details
freeze 2 (15.26 KB, text/x-log)
2022-01-27 03:50 EST, Andrey Loskutov CLA
no flags Details
Console output with verbose info before hang (1.85 MB, text/plain)
2022-01-28 04:14 EST, Andrey Loskutov CLA
no flags Details
Console output with verbose info for successfull build (5.14 MB, text/plain)
2022-01-28 05:22 EST, Hannes Wellmann CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Hannes Wellmann CLA 2022-01-27 03:41:05 EST
Recently I observed that the PDE.UI Jenkins build-job sometimes freezes and finally times out when running the org.eclipse.pde.api.tools.tests.ApiToolsPluginTestSuite.

Just with my latest Gerrit-Change I got the freez two times with the same Commit and the thrid build finally succeeded.

Thanks to Bug 578391 I can now provide the StackTraces below.


Running org.eclipse.pde.api.tools.tests.ApiToolsPluginTestSuite
reflectNestedClassUseDollar=true due to isJRE9Plus
Possible frozen test case
"main": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.jdt.internal.core.search.processing.JobManager.performConcurrentJob(JobManager.java:284)
    org.eclipse.jdt.internal.core.search.BasicSearchEngine.searchAllSecondaryTypeNames(BasicSearchEngine.java:1713)
    org.eclipse.jdt.internal.core.JavaModelManager.secondaryTypesSearching(JavaModelManager.java:4958)
    org.eclipse.jdt.internal.core.JavaModelManager.secondaryTypes(JavaModelManager.java:4799)
    org.eclipse.jdt.internal.core.NameLookup.findSecondaryType(NameLookup.java:712)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:1000)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:823)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:781)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:1074)
    org.eclipse.jdt.internal.core.JavaProject.findType(JavaProject.java:1691)
    org.eclipse.jdt.internal.core.JavaProject.findType(JavaProject.java:1786)
    org.eclipse.jdt.internal.core.JavaProject.findType(JavaProject.java:1684)
    org.eclipse.pde.api.tools.internal.builder.AbstractProblemDetector.createProblem(AbstractProblemDetector.java:183)
    org.eclipse.pde.api.tools.internal.builder.AbstractProblemDetector.createProblems(AbstractProblemDetector.java:523)
    org.eclipse.pde.api.tools.internal.builder.ReferenceAnalyzer.analyze(ReferenceAnalyzer.java:258)
    org.eclipse.pde.api.tools.internal.builder.BaseApiAnalyzer.checkApiUsage(BaseApiAnalyzer.java:1286)
    org.eclipse.pde.api.tools.internal.builder.BaseApiAnalyzer.analyzeComponent(BaseApiAnalyzer.java:289)
    org.eclipse.pde.api.tools.internal.builder.ApiAnalysisBuilder.buildAll(ApiAnalysisBuilder.java:922)
    org.eclipse.pde.api.tools.internal.builder.ApiAnalysisBuilder.work(ApiAnalysisBuilder.java:422)
    org.eclipse.pde.api.tools.internal.builder.ApiAnalysisBuilder.build(ApiAnalysisBuilder.java:406)
    org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:856)
    org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
    org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:232)
    org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:281)
    org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:334)
    org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
    org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:337)
    org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:392)
    org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:413)
    org.eclipse.core.internal.resources.Workspace.buildInternal(Workspace.java:516)
    org.eclipse.core.internal.resources.Workspace.build(Workspace.java:406)
    org.eclipse.jdt.core.tests.builder.TestingEnvironment.fullBuild(TestingEnvironment.java:552)
    org.eclipse.jdt.core.tests.builder.BuilderTests.fullBuild(BuilderTests.java:413)
    org.eclipse.pde.api.tools.builder.tests.usage.UsageTest.deployUsageTest(UsageTest.java:125)
    org.eclipse.pde.api.tools.builder.tests.usage.FieldUsageTests.x1(FieldUsageTests.java:91)
    org.eclipse.pde.api.tools.builder.tests.usage.FieldUsageTests.testFieldUsage1F(FieldUsageTests.java:67)
    java.base@11.0.2/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base@11.0.2/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base@11.0.2/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base@11.0.2/java.lang.reflect.Method.invoke(Method.java:566)
    junit.framework.TestCase.runTest(TestCase.java:177)
    org.eclipse.jdt.core.tests.junit.extension.TestCase.runTest(TestCase.java:969)
    junit.framework.TestCase.runBare(TestCase.java:142)
    junit.framework.TestResult$1.protect(TestResult.java:122)
    junit.framework.TestResult.runProtected(TestResult.java:142)
    junit.framework.TestResult.run(TestResult.java:125)
    junit.framework.TestCase.run(TestCase.java:130)
    junit.framework.TestSuite.runTest(TestSuite.java:241)
    junit.framework.TestSuite.run(TestSuite.java:236)
    junit.framework.TestSuite.runTest(TestSuite.java:241)
    junit.framework.TestSuite.run(TestSuite.java:236)
    junit.framework.TestSuite.runTest(TestSuite.java:241)
    junit.framework.TestSuite.run(TestSuite.java:236)
    org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:90)
    org.junit.runners.Suite.runChild(Suite.java:128)
    org.junit.runners.Suite.runChild(Suite.java:27)
    org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
    org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
    org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
    org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
    org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
    org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
    org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
    org.junit.runners.ParentRunner.run(ParentRunner.java:413)
    org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    org.junit.runner.JUnitCore.run(JUnitCore.java:115)
    org.junit.vintage.engine.execution.RunnerExecutor.execute(RunnerExecutor.java:42)
    org.junit.vintage.engine.VintageTestEngine.executeAllChildren(VintageTestEngine.java:80)
    org.junit.vintage.engine.VintageTestEngine.execute(VintageTestEngine.java:72)
    org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
    org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
    org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
    org.junit.platform.launcher.core.EngineExecutionOrchestrator$$Lambda$342/0x0000000100484440.accept(Unknown Source)
    org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
    org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
    org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
    org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
    org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
    org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:53)
    org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invokeAllTests(JUnitPlatformProvider.java:150)
    org.apache.maven.surefire.junitplatform.JUnitPlatformProvider.invoke(JUnitPlatformProvider.java:124)
    java.base@11.0.2/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base@11.0.2/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base@11.0.2/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base@11.0.2/java.lang.reflect.Method.invoke(Method.java:566)
    org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray2(ReflectionUtils.java:206)
    org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:161)
    org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:84)
    org.eclipse.tycho.surefire.osgibooter.OsgiSurefireBooter.run(OsgiSurefireBooter.java:128)
    org.eclipse.tycho.surefire.osgibooter.HeadlessTestApplication.start(HeadlessTestApplication.java:23)
    org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:203)
    org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:136)
    org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
    org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:401)
    org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:255)
    java.base@11.0.2/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base@11.0.2/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base@11.0.2/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base@11.0.2/java.lang.reflect.Method.invoke(Method.java:566)
    app//org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:659)
    app//org.eclipse.equinox.launcher.Main.basicRun(Main.java:596)
    app//org.eclipse.equinox.launcher.Main.run(Main.java:1467)
    app//org.eclipse.equinox.launcher.Main.main(Main.java:1440)

"Reference Handler": RUNNABLE
    java.base@11.0.2/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
    java.base@11.0.2/java.lang.ref.Reference.processPendingReferences(Reference.java:241)
    java.base@11.0.2/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:213)

"Finalizer": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
    java.base@11.0.2/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
    java.base@11.0.2/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:170)

"Signal Dispatcher": RUNNABLE

"Common-Cleaner": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
    java.base@11.0.2/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:148)
    java.base@11.0.2/java.lang.Thread.run(Thread.java:834)
    java.base@11.0.2/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:134)

"Active Thread: Equinox Container: cd50eac4-d09c-4e0d-b1ec-28f010cb85f4": TIMED_WAITING
    java.base@11.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    java.base@11.0.2/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:234)
    java.base@11.0.2/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2123)
    java.base@11.0.2/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
    java.base@11.0.2/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
    java.base@11.0.2/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1054)
    java.base@11.0.2/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1114)
    java.base@11.0.2/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    java.base@11.0.2/java.lang.Thread.run(Thread.java:834)

"Framework Event Dispatcher: Equinox Container: cd50eac4-d09c-4e0d-b1ec-28f010cb85f4": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:394)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:335)

"Start Level: Equinox Container: cd50eac4-d09c-4e0d-b1ec-28f010cb85f4": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:394)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:335)

"Bundle File Closer": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:394)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:335)

"Refresh Thread: Equinox Container: cd50eac4-d09c-4e0d-b1ec-28f010cb85f4": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.getNextEvent(EventManager.java:394)
    org.eclipse.osgi.framework.eventmgr.EventManager$EventThread.run(EventManager.java:335)

"SCR Component Actor": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.apache.felix.scr.impl.ComponentActorThread.run(ComponentActorThread.java:83)
    java.base@11.0.2/java.lang.Thread.run(Thread.java:834)

"EMF Reference Cleaner": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
    java.base@11.0.2/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
    org.eclipse.emf.common.util.CommonUtil$1ReferenceClearingQueuePollingThread.run(CommonUtil.java:70)

"Worker-JM": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.core.internal.jobs.InternalWorker.run(InternalWorker.java:61)

"Worker-0": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
    org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:242)
    org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

"Worker-1": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
    org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:258)
    org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

"Java indexing": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:416)
    java.base@11.0.2/java.lang.Thread.run(Thread.java:834)

"Worker-2": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
    org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:242)
    org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

"Worker-3": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
    org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:242)
    org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

"Worker-4: FreezeMonitor": RUNNABLE
    java.management@11.0.2/sun.management.ThreadImpl.getThreadInfo1(Native Method)
    java.management@11.0.2/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:190)
    org.eclipse.pde.ui.tests.util.FreezeMonitor$1.run(FreezeMonitor.java:48)
    org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

"Worker-5": TIMED_WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.core.internal.jobs.WorkerPool.sleep(WorkerPool.java:200)
    org.eclipse.core.internal.jobs.WorkerPool.startJob(WorkerPool.java:258)
    org.eclipse.core.internal.jobs.Worker.run(Worker.java:58)

"ForkJoinPool.commonPool-worker-3": WAITING
    java.base@11.0.2/jdk.internal.misc.Unsafe.park(Native Method)
    java.base@11.0.2/java.util.concurrent.locks.LockSupport.park(LockSupport.java:194)
    java.base@11.0.2/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1628)
    java.base@11.0.2/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:177)


Build timed out (after 41 minutes). Marking the build as aborted.
Terminating xvnc.
Comment 1 Hannes Wellmann CLA 2022-01-27 03:43:15 EST
@Andrey can you help in this regard respectively can you get sufficient information from the StackTrace to identify the problem?
Comment 2 Andrey Loskutov CLA 2022-01-27 03:49:47 EST
Created attachment 287910 [details]
freeze 1
Comment 3 Andrey Loskutov CLA 2022-01-27 03:50:05 EST
Created attachment 287911 [details]
freeze 2
Comment 4 Andrey Loskutov CLA 2022-01-27 04:28:28 EST
Looks like a hang in JDT search code, waiting for indexer.
Almost same stack as in bug 574647.

Important parts: 


Main thread waiting:

    java.base@11.0.2/java.lang.Object.wait(Native Method)
    org.eclipse.jdt.internal.core.search.processing.JobManager.performConcurrentJob(JobManager.java:284)
    org.eclipse.jdt.internal.core.search.BasicSearchEngine.searchAllSecondaryTypeNames(BasicSearchEngine.java:1713)
    org.eclipse.jdt.internal.core.JavaModelManager.secondaryTypesSearching(JavaModelManager.java:4958)
    org.eclipse.jdt.internal.core.JavaModelManager.secondaryTypes(JavaModelManager.java:4799)
    org.eclipse.jdt.internal.core.NameLookup.findSecondaryType(NameLookup.java:712)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:1000)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:823)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:781)
    org.eclipse.jdt.internal.core.NameLookup.findType(NameLookup.java:1074)
    org.eclipse.jdt.internal.core.JavaProject.findType(JavaProject.java:1691)
    org.eclipse.jdt.internal.core.JavaProject.findType(JavaProject.java:1786)
    org.eclipse.jdt.internal.core.JavaProject.findType(JavaProject.java:1684)


Indexer job waiting too:

"Java indexing": WAITING
    java.base@11.0.2/java.lang.Object.wait(Native Method)
    java.base@11.0.2/java.lang.Object.wait(Object.java:328)
    org.eclipse.jdt.internal.core.search.processing.JobManager.run(JobManager.java:416)
    java.base@11.0.2/java.lang.Thread.run(Thread.java:834)


What is interesting, this is random failure. 
The problem might occur if some (bad written) test randomly disables indexer (org.eclipse.jdt.internal.core.search.processing.JobManager.disable()) and fails to re-enable it again (on some error or fail condition), and the indexer stays disabled (it has a counter) - which causes the test in question (FieldUsageTests) wait forever because it needs indexer working.
Comment 5 Hannes Wellmann CLA 2022-01-28 02:38:27 EST
(In reply to Andrey Loskutov from comment #4)
> 
> What is interesting, this is random failure. 
> The problem might occur if some (bad written) test randomly disables indexer
> (org.eclipse.jdt.internal.core.search.processing.JobManager.disable()) and
> fails to re-enable it again (on some error or fail condition), and the
> indexer stays disabled (it has a counter) - which causes the test in
> question (FieldUsageTests) wait forever because it needs indexer working.

Its indeed interesting.
I searched the entire pde.ui and pde.build code base in my workspace and found no calls the mentioned JobManager.disable() method.

With the stack-trace and from looking into the code of the JobManager the very root cause seems to be that awaitingJobsCount() always returns a value greater zero. Which could be because the JobManager is not activated or because the List of awaitingJobs never becomes empty. I don't know if the former is realistic? If I didn't oversaw something it looks like elements from awaitingJobs are only removed in moveToNextJob() which is only called in the JobManager's run method. I assume that's the reason that you suspect that the JobManager is disabled? But could there another reason?

It could maybe also help to make the JobManager verbose. I'm about to prepare a (temporary) Gerrit for this. Maybe we are 'lucky' and this change immediately hits the freeze.
Comment 6 Eclipse Genie CLA 2022-01-28 02:40:08 EST
New Gerrit change created: https://git.eclipse.org/r/c/pde/eclipse.pde.ui/+/190093
Comment 7 Andrey Loskutov CLA 2022-01-28 04:14:48 EST
Created attachment 287920 [details]
Console output with verbose info before hang
Comment 8 Andrey Loskutov CLA 2022-01-28 04:17:25 EST
(In reply to Andrey Loskutov from comment #7)
> Created attachment 287920 [details]
> Console output with verbose info before hang

Gayan, could you please take a look on the attached indexer output before the hang (and the hang stack that follows), may be you see something obvious?

@Hannes: I must confess, I have not much time to investigate deeper, but enabling verbose output is surely helpful.
Comment 9 Hannes Wellmann CLA 2022-01-28 05:22:11 EST
Created attachment 287921 [details]
Console output with verbose info for successfull build
Comment 10 Hannes Wellmann CLA 2022-01-28 05:25:16 EST
(In reply to Andrey Loskutov from comment #8)
> (In reply to Andrey Loskutov from comment #7)
> > Created attachment 287920 [details]
> > Console output with verbose info before hang
> 
> Gayan, could you please take a look on the attached indexer output before
> the hang (and the hang stack that follows), may be you see something obvious?
> 

I also added the console log of the successful build for comparison.


> @Hannes: I must confess, I have not much time to investigate deeper, but
> enabling verbose output is surely helpful.

Understand that. My problem is that I'm not familiar with that part of the code. I can try to dig a bit into it, but it would probably be much more efficient/faster if somebody with more experience in this area would have a look at it.
Comment 11 Gayan Perera CLA 2022-01-28 16:22:50 EST
@Hannes i still couldn’t find the time to look at the traces in detail. But do you see that the infinite wait happen at IndexManager.waitForIndex method ?
Comment 12 Hannes Wellmann CLA 2022-01-30 08:43:15 EST
(In reply to Gayan Perera from comment #11)
> @Hannes i still couldn’t find the time to look at the traces in detail. But
> do you see that the infinite wait happen at IndexManager.waitForIndex method
> ?

I cannot say more that I did not found any occurrence of "waitForIndex" in the logs or stack-traces.
Comment 14 Gayan Perera CLA 2022-03-11 14:49:39 EST
The following is interesting

Thread[main,5,main] DISABLING background indexing
Thread[main,5,main] -> index state updated to: REBUILDING for: file:/home/jenkins/agent/workspace/eclipse.pde.ui-Gerrit/apitools/org.eclipse.pde.api.tools.tests/target/work/data/.metadata/.plugins/org.eclipse.jdt.core/2038840533.index
Thread[main,5,main] REQUEST   background job - indexing project /usagetests
Thread[main,5,main] AWAITING JOBS count: 7
Thread[main,5,main] ENABLING  background indexing
reflectNestedClassUseDollar=true due to isJRE9Plus
Thread[main,5,main] REQUEST   background job - indexing /usagetests/src/x/y/z/testF1.java
Thread[main,5,main] AWAITING JOBS count: 8
Thread[main,5,main] STARTING  concurrent job - searching SecondaryTypeDeclarationPattern: pkg<*>, enclosing<*>, type<*>, exact match, case sensitive, generic full match, fine grain: none
Thread[main,5,main] -> selected 2 indexes out of total indexes 2 after qualify filtering - org.eclipse.jdt.internal.core.search.IndexSelector@302d6539
Possible frozen test case
"main": TIMED_WAITING


Seems like this enabling and disabling of background jobs seems to have a issue of awaiting jobs. Seems like after each execution the awaiting jobs increases.
Comment 15 Eclipse Genie CLA 2024-03-01 00:27:36 EST
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.