Community
Participate
Working Groups
Build Identifier: 3.4.2 We have code to search annotations from projects by using JDT search API. For one of our workspace, we hit OOM error, the memory analyzer shows that "org.eclipse.jdt.internal.compiler.ast.TypeDeclaration" and "org.eclipse.jdt.internal.core.search.matching.PossibleMatch" are the 2 top objects. The API we are using for the Java search is: org.eclipse.jdt.core.search.SearchEngine.search(SearchPattern, SearchParticipant[], IJavaSearchScope, SearchRequestor, IProgressMonitor) We created an annotation search pattern. (about 20 annotations) This is the stack from dump file: 3XMTHREADINFO "Worker-26" J9VMThread:0x48D75B00, j9thread_t:0x4A0EE4EC, java/lang/Thread:0x0ABA48F0, state:R, prio=1 3XMTHREADINFO1 (native thread ID:0xE9C, native priority:0x1, native policy:UNKNOWN) 3XMTHREADINFO3 Java callstack: 4XESTACKTRACE at org/eclipse/jdt/internal/compiler/parser/Parser.getUnspecifiedReferenceOptimized(Bytecode PC:79(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocatorParser.getUnspecifiedReferenceOptimized(Bytecode PC:1(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/compiler/parser/Parser.consumeInsideCastExpressionLL1(Bytecode PC:2(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/compiler/parser/Parser.consumeRule(Bytecode PC:4272(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/compiler/parser/Parser.parse(Bytecode PC:464(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/compiler/parser/Parser.parse(Bytecode PC:108(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocatorParser.parseBodies(Bytecode PC:115(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocatorParser.parseBodies(Bytecode PC:38(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocator.getMethodBodies(Bytecode PC:107) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocator.parseAndBuildBindings(Bytecode PC:155(Compiled Code)) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocator.locateMatches(Bytecode PC:73) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocator.locateMatches(Bytecode PC:81) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/matching/MatchLocator.locateMatches(Bytecode PC:645) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/JavaSearchParticipant.locateMatches(Bytecode PC:41) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/BasicSearchEngine.findMatches(Bytecode PC:404) 4XESTACKTRACE at org/eclipse/jdt/internal/core/search/BasicSearchEngine.search(Bytecode PC:20) 4XESTACKTRACE at org/eclipse/jdt/core/search/SearchEngine.search(Bytecode PC:11) 4XESTACKTRACE at com/ibm/etools/annotations/core/internal/search/JavaSearchUtils.findAnnotations(Bytecode PC:165) Reproducible: Always Steps to Reproduce: 1. Import the zip file into the workspace.
Created attachment 162064 [details] Memory report from MAT (use Internet Explorer to view it)
In order to help us to reproduce, please: 1) provide the zip you referred to in comment 0: > Steps to Reproduce: > 1. Import the zip file into the workspace. 2) provide a snippet of code to show us how you create the search pattern and what are the exact values of the arguments when you call the search(...) API method TIA
The Java code in the workspace was generated by a third party product, and has the characteristic of some very large Java files. The biggest one is 2.5 MB. They can have lots of fields, and large methods. One method that I looked at went on for 28 pages. I can provide you with an archive file and/or a crash dump. (I find using MAT and the crash dump is an excellent way of seeing everything that is in memory).
Gary, please make sure that you provide to Frédéric and Satyam all required material to reproduce this issue. Having a MAT dump can only show the issue. Reproducing it is critical to fix it.
(In reply to comment #4) I sent them both the workspace and full dump yesterday.
(In reply to comment #5) > (In reply to comment #4) > I sent them both the workspace and full dump yesterday. I can reproduce the OOME with the provided material. Note that the dump is not very useful as I cannot read it, but now I can do it by myself. So, I'm currently investigating and I'll be back soon with feedback about this...
I need some more informations: 1) what are the -Xmx value used to run the VM? 2) is there a way to know what is exactly the pattern used for the search comment 0 says: "annotation search pattern. (about 20 annotations)", but I need to recreate it on my side. Hence, would it be possible to build
(In reply to comment #7) > I need some more informations: > 1) what are the -Xmx value used to run the VM? -Xmx1024M
(In reply to comment #7) > I need some more informations: > 1) what are the -Xmx value used to run the VM? > 2) is there a way to know what is exactly the pattern used for the search > comment 0 says: "annotation search pattern. (about 20 annotations)", > but I need to recreate it on my side. Hence, would it be possible to build sorry, an unexpected commit... Let's rephrase and finish the last sentence: Hence, would it be possible to know what are all the fully qualified names used to build the pattern? Then I could recreate it as a string pattern and have a good simulation of the context when the OOME occurred. TIA
Could you also please let us know the exact version of the JDT/Core plugin (org.eclipse.jdt.core) which is in use in the product? TIA
Here are the conclusion on this issue. The Search Engine does have problems while searching for annotation type reference using the pattern provided from RAD code. But these problems are not performance ones, only troubles with some weird anonymous declaration (see bug 306196) and with unexpected matches (see bug 306223). Hacking the code to rapidly get rid of these two issues, I was always able to search for all annotations references in the provided workspace without any OOME. Hence, my guess is that the OOME occurring using RAD should be a combination of: 1) By keeping the annotations in a list, the collector may consume a significant amount of memory if there are a large number of matches and if the stored annotations have already been opened by any other operation. 2) Basically, RAD also should also consume a significant amount of memory which is obviously free in my workspace using a "pure" Eclipse, hence may explain that I cannot reproduce the OOME Note that the RAD code I saw to search for annotations references seems to be familiar to me and going down into my archives, I found that a bug against search for a NPE while using similar code was already opened two years ago (see bug 215684) Note that at that time, I said that creating a simple reference pattern for ANNOTATION_TYPE was sub-optimal although the fine grain was added in 3.4... I suggested to use a "annotation type reference" fine grain flag instead to speed-up the search. I provided a full sample to show how to use it (see bug 215684 comment 6). So I would have the same advice today: to find all annotations used as annotations in a scope, it would be better to create the pattern as follow: SearchPattern pattern1 = SearchPattern.createPattern(stringPattern, IJavaSearchConstants.ANNOTATION_TYPE, IJavaSearchConstants.ANNOTATION_TYPE_REFERENCE, SearchPattern.R_EXACT_MATCH); Satyam made some tests using a JDT/Core plugin hacked to force the usage of such a pattern in this peculiar case (search pattern with 'Search for'=ANNOTATION_TYPE and 'Limit To'=REFERENCES automatically replace with a pattern with 'Limit To'=ANNOTATION_TYPE_REFERENCE) and he was able to use RAD on the given workspace without any OOME and good responsiveness... So please confirm ASAP that changing the used pattern fixes the issue and I'll close this bug as NOT_ECLIPSE Thanks
Benson sent me a patch, which I believe followed the advice in comment 11. The patch worked. AFAIK, we will be able to close this. But since I don't know exactly what Benson, patched, I'll leave it to him, to close/update this defect. (Benson has been on vacation this week, which is why there has been a little bit of a delay in responding).
(In reply to comment #12) > Benson sent me a patch, which I believe followed the advice in comment 11. The > patch worked. AFAIK, we will be able to close this. > > But since I don't know exactly what Benson, patched, I'll leave it to him, to > close/update this defect. > > (Benson has been on vacation this week, which is why there has been a little > bit of a delay in responding). Thanks for the feedback Gary. I turn off this bug to RESOLVED status as NOT_ECLIPSE due to the fact that it was a change in the client code which fixes the original issue. Benson, Please set it as VERIFIED if you agree with the whole history, TIA
