Bug 579204 - MetaIndex: incomplete type hierarchy
Summary: MetaIndex: incomplete type hierarchy
Status: ASSIGNED
Alias: None
Product: JDT
Classification: Eclipse Project
Component: Core (show other bugs)
Version: 4.21   Edit
Hardware: All All
: P3 major (vote)
Target Milestone: ---   Edit
Assignee: Gayan Perera CLA
QA Contact:
URL:
Whiteboard: stalebug
Keywords: regression
Depends on:
Blocks:
 
Reported: 2022-03-10 09:50 EST by Jörg Kubitz CLA
Modified: 2024-03-23 19:21 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jörg Kubitz CLA 2022-03-10 09:50:57 EST
With a fresh workspace in 4.22 we see incomplete type hierarchy.

Starting instead
 eclipse.exe -vmargs -Dorg.eclipse.jdt.disableMetaIndex=true
the problem is gone. => regression from Bug 570078

We use 
 org.eclipse.jdt.core.IType.newTypeHierarchy(IProgressMonitor)

That call should not return before all necessary results are computed.
It even waits for index (like 4 times longer then in 4.20) but still does not find the desired result:

        at java.lang.Object.wait(java.base@11.0.14/Native Method)
        - waiting on <no object reference available>
        at org.eclipse.jdt.internal.core.search.processing.JobManager.performConcurrentJob(JobManager.java:284)
        - waiting to re-lock in wait() <0x0000000702b5f228> (a org.eclipse.jdt.internal.core.search.indexing.IndexManager)
        at org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.legacySearchAllPossibleSubTypes(IndexBasedHierarchyBuilder.java:605)
        at org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.searchAllPossibleSubTypes(IndexBasedHierarchyBuilder.java:506)
        at org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.determinePossibleSubTypes(IndexBasedHierarchyBuilder.java:465)
        at org.eclipse.jdt.internal.core.hierarchy.IndexBasedHierarchyBuilder.build(IndexBasedHierarchyBuilder.java:158)
        at org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy.compute(TypeHierarchy.java:323)
        at org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy.refresh(TypeHierarchy.java:1319)
        - locked <0x000000073dd572e8> (a org.eclipse.jdt.internal.core.hierarchy.TypeHierarchy)
        at org.eclipse.jdt.internal.core.CreateTypeHierarchyOperation.executeOperation(CreateTypeHierarchyOperation.java:94)
        at org.eclipse.jdt.internal.core.JavaModelOperation.run(JavaModelOperation.java:740)
        at org.eclipse.jdt.internal.core.JavaModelOperation.runOperation(JavaModelOperation.java:806)
        at org.eclipse.jdt.internal.core.SourceType.newTypeHierarchy(SourceType.java:950)
        at org.eclipse.jdt.internal.core.SourceType.newTypeHierarchy(SourceType.java:907)


The problem occurs only with a fresh workspace. After some time the problem goes away - after the indexer thread becomes idle.

The result is missing some xtext autogenerated java files - that already have been build with jdt at the time we call newTypeHierarchy(). it did work in 4.20
Comment 1 Gayan Perera CLA 2022-03-10 14:39:43 EST
@Jörg do you have a way to reproduce this with some open source project which i can clone and test with, without needing to setup a xtext workspace ?
Comment 2 Gayan Perera CLA 2022-03-10 15:19:21 EST
I think one possibility is that

org.eclipse.jdt.internal.core.search.indexing.IndexManager.MetaIndexUpdateRequest.execute(IProgressMonitor)

The above request run once and update the metaindex and other consecutive updates could get cancelled. If its possible can you check if it does get cancelled in your scenario ?
Comment 3 Jörg Kubitz CLA 2022-03-11 03:11:11 EST
(In reply to Gayan Perera from comment #2)

It's very hard for me to reproduce that issue, so i do not have a trivial example yet.

> MetaIndexUpdateRequest

I see it is "removed" in cases like this:

 indexManager.disable();
 try {
 	indexManager.discardJobs(null); // deletes *all* jobs
 	...
 } finally {
 	indexManager.enable();
 }

Also - Is it bad that "MetaIndexUpdateRequest" is not the last in JobManager.awaitingJobs? What if it runs before the other index jobs? Feels wrong.

Also - i dislike MetaIndexUpdateRequest.equals() is based on non final field "isCancelled" - may be it's working at the moment, but if one would change awaitingJobs to a Set implementation it would be crucial that a.equals(b) does not change during livetime
Comment 4 Jörg Kubitz CLA 2022-03-11 04:03:48 EST
I also think it is wrong, that IndexManager.scheduleDocumentIndexing() does not directly add a MetaIndexUpdateRequest but deffers that until execute(). I guess the index queue can get meanwhile empty making the newTypeHierarchy() return and the MetaIndexUpdateRequest may be added a blink of an eye too late.
Comment 5 Gayan Perera CLA 2022-03-11 14:41:56 EST
Good points @Jorg,

I think some of the changes @Andrey and Myself did with some differing was to avoid deadlocks. But i remember later we did some improvements (I think it was you) to improve the JobManager.

May be we should have a look at them. Will you have time to look or do you have some suggestions how we can improve it ?

I would really like the idea of running the metaindex update request as a post index task just before releasing the locks.
Comment 6 Gayan Perera CLA 2022-03-16 15:04:25 EDT
I will start a improvement for this by maintaining metaindex state and checking that before start searching. We could treat it as a normal index and if its not updated we can rebuild that since we have required information on the normal indexes. That should solve the issue.

I also found the same issue Jorg mentioned with groovy as well since i was working on some jenkins pipeline this week.
Comment 7 Eclipse Genie CLA 2022-03-18 16:07:17 EDT
New Gerrit change created: https://git.eclipse.org/r/c/jdt/eclipse.jdt.core/+/192070
Comment 8 Gayan Perera CLA 2022-03-18 16:08:54 EDT
@Jorg can you check the gerrit if possible and see if it solves your xtext problem ? i will work on another gerrit to rebuild the metaindex in case we find its not in correct state. But before that i would like to see if the problem you found will be solved by this small change ?
Comment 9 Jörg Kubitz CLA 2022-03-23 02:25:42 EDT
(In reply to Gayan Perera from comment #8)
> @Jorg can you check the gerrit if possible and see if it solves your xtext

Gayan, i am sorry, since i can not 100% reproduce the issue i cant verify if it's solved. Even though i spend a lot of time in debugging i could not find THE single reason it failed.
Comment 10 Eclipse Genie CLA 2024-03-23 19:21:05 EDT
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet.

If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant.

--
The automated Eclipse Genie.