Community
Participate
Working Groups
I am using a workspace containing several projects (libraries and executable). Until recently I used Eclipse CDT in the Indigo release and all works fine. Recently I tried to update to the current Juno release and got an error everytime the indexer runs: An internal error occurred during: "Update Monitor". Java heap space Starting eclipse with the -vmargs -Xmx1024m option does not improve the situation. So I switched back to indigo for now. Removing the include paths to Qt in the C/C++ General/Path and Symbols section removes this problem, but is not an option because I like to have Qt in the index. I tried to reproduce this problem in a small workspace containing a single file and using Qt, but there the problem does not occur. As the big workspace contain all our company code, I am sorry, but I cannot provide the workspace for you to reproduce the problem.
java.lang.OutOfMemoryError: Java heap space Dialog open exception: java.lang.NullPointerException at org.eclipse.ui.internal.ide.IDEWorkbenchErrorHandler.openQuestionDialog(IDEWorkbenchErrorHandler.java:195) at org.eclipse.ui.internal.ide.IDEWorkbenchErrorHandler.handleException(IDEWorkbenchErrorHandler.java:154) at org.eclipse.ui.internal.ide.IDEWorkbenchErrorHandler.access$0(IDEWorkbenchErrorHandler.java:146) at org.eclipse.ui.internal.ide.IDEWorkbenchErrorHandler$1.runInUIThread(IDEWorkbenchErrorHandler.java:121) at org.eclipse.ui.progress.UIJob$1.run(UIJob.java:95) at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35) at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:135) at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:4144) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3761) at org.eclipse.swt.widgets.Display.release(Display.java:3814) at org.eclipse.swt.graphics.Device.dispose(Device.java:295) at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:140) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:353) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:180) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:629) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:584) at org.eclipse.equinox.launcher.Main.run(Main.java:1438)
Can you use Memory Analyser http://www.eclipse.org/mat/ to look what objects take most memory or just acquire heap dump?
The top components listed by Eclipse Memory Analyzer are: org.eclipse.ctd.core (43%) org.eclipse.core.jobs (37%) org.eclipse.ctd.ui (10%) Is this what you want to know? The .hprof file is around 300 MB. If you like I can attach it here, if not too big, or copy to any ftp server.
I'd be interested to take a look. I suppose if bugzilla lets you attach it it is not against the policy. If not I can take from ftp server or you can send it to me privately.
Created attachment 226746 [details] The heap dump as a zip archive
I can confirm this issue. I'm having exactly the same error in Juno. Increasing the heap size does not resolve anything. Also removing Qt includes solves the problem, but also this is not an option as I don't have Qt indexed then. In eclipse 3.8 everything is working fine, so I'll keep using it until this can be resolved. I am using Debian x64 with OpenJDK 1.6
(In reply to comment #5) > Created attachment 226746 [details] > The heap dump as a zip archive This heapdump does not indicate problems with heap. There is only 250Mb of heap used total, nothing out of ordinary.
I created this heapdump by starting eclipse as follows. c:/Program\ Files\ \(x86\)/eclipse-juno/eclipse -vmargs -XX:+HeapDumpOnOutOfMemoryError & I just repeated this step and got a heap dump of similar size. I could upload this again, but I fear that it would be similar.
Add -Xmx1024m option, otherwise we may not even get to the real problem. The bigger the heapdump the better for troubleshooting.
Created attachment 226983 [details] Heapdump with -Xmx1024m option
(In reply to comment #10) > Heapdump with -Xmx1024m option It appears that instances of CPPASTName hold over 500Mb of heap space here... With related objects it is over 800Mb of heap. Everything else appears normal. Class Name | Objects | Shallow Heap | Retained Heap --------------------------------------------------------------------------------------------------- org.eclipse.cdt.internal.core.dom.parser.cpp.CPPASTName| 1,854,409 | 89,011,632 | >= 552,818,640 --------------------------------------------------------------------------------------------------- Well, somebody with better expertise in indexer/parser than me should take it from here.
(In reply to comment #11) When one of the first indexed source files directly or indirectly includes many header files, all that volume of code has to be parsed together. To verify how large the code is you can pass the source file causing stack overflow through the C++ preprocessor. 1.8 million is a lot of names. With 4 names per line this is equivalent to 450000 lines of code. Compare your preprocessed file to that and try to estimate the number of names in it.
This may be, It is a quite big workspace containing several libraries and executables with a lot of files. How should I detect the one that causes the overflow? By the way does Juno need so much more memory than indigo? indigo runs without an -Xmx1024m argument, so it seems to be still quite modest with memory.
(In reply to comment #13) > This may be, It is a quite big workspace containing several libraries and > executables with a lot of files. Total size of the project doesn't matter as much as the number of header files included directly or indirectly into one of the first files being indexed. > How should I detect the one that causes the overflow? Watch Progress View for the file being indexed when Eclipse runs out of memory.
I have located the file where the indexer runs out of memory. After running the MSVC preprocessor and removing empty lines from the output I get a file of 161588 lines. If I apply your rule of thumb, 4 names per line, I would get around 600.000 names. Additionally I observed that the file contains a lot of template code.
(In reply to comment #15) Hmm, 600K names is more than 3 times less than 1854K names reported by the profiler. Can you attempt to get a more accurate name count by replacing all nonalphanumeric characters with spaces and counting the number of words after that?
I tried to get a more accurate count, by 1) Removing non alphanumeric characters sed 's/[^a-zA-Z0-9]/ /g' prepro-clean.txt > prepro-clean2.txt 2) Replace white space by newline sed 's/\s\+/\n/g' prepro-clean2.txt > prepro-clean3.txt 3) Count lines wc -l prepro-clean3.txt => 1301914 4) Remove duplicates sort -f prepro-clean3.txt | uniq > prepro-clean4.txt 5) Count lines without duplicates wc -l prepro-clean4.txt => 11328
(In reply to comment #17) > I tried to get a more accurate count, by > > 1) Removing non alphanumeric characters > sed 's/[^a-zA-Z0-9]/ /g' prepro-clean.txt > prepro-clean2.txt You also need to replace C/C++ keywords with spaces. > 2) Replace white space by newline > sed 's/\s\+/\n/g' prepro-clean2.txt > prepro-clean3.txt > 3) Count lines > wc -l prepro-clean3.txt => 1301914 > > > 4) Remove duplicates > sort -f prepro-clean3.txt | uniq > prepro-clean4.txt Each name occurrence requires a separate object. No need to remove duplicates. > 5) Count lines without duplicates > wc -l prepro-clean4.txt => 11328
> You also need to replace C/C++ keywords with spaces. How should I do this ?
> Each name occurrence requires a separate object. No need to remove > duplicates. Why? This sounds like a defect in the indexer, if it has to create separate objects for the same identifiers. And the question of why such a regression in Juno vs. Indigo still stands big time. P.S. I have the same problem -- the code that Indigo indexer was able to index without a problem now causes Juno indexer to run out of memory.
(In reply to comment #20) > > Each name occurrence requires a separate object. No need to remove > > duplicates. > > Why? This sounds like a defect in the indexer, if it has to create separate > objects for the same identifiers. Each name contains unique location information that is later stored in the index. This location information is used by C/C++ search, call hierarchy, etc.
(In reply to comment #21) > Each name contains unique location information that is later stored in the > index. This location information is used by C/C++ search, call hierarchy, > etc. Got it. Still, there is an obvious regression in Juno, regarding the memory utilization by the Indexer. I've given Juno 2gb heap, and it still could not finish what Indigo could with just 1gb heap. (Going back to these records, I would assume that 99% of identifiers in a translation unit are duplicates. Does the entire record have to be replicated, or is there lots of common information that can be shared by those records?)
I had the same problem with the indexer consuming up to more than 30GB of memory without progressing (always at the same percentage) before I had to kill it. A workaround for me was decreasing the indexers absolute cache limits from 64MB to 32MB, though I don't know if this will work in general. You may find the settings at Window -> Preferences -> C/C++ -> Indexer
(In reply to comment #22) > Still, there is an obvious regression in Juno, regarding the memory > utilization by the Indexer. I've given Juno 2gb heap, and it still could > not finish what Indigo could with just 1gb heap. How does indexer statistics (written to error log on rebuild) compares between Juno and Indigo? > (Going back to these records, I would assume that 99% of identifiers in a > translation unit are duplicates. Does the entire record have to be > replicated, or is there lots of common information that can be shared by > those records?) If you look at CPPASTName and the classes it extends, you will find that only a small fraction of data could be shared.
Created attachment 229219 [details] Options file to enable indexer tracing I'm also running OutOfMemory with CDT 8.1.2 and it looks like the BOOST library is involved somehow in my case. It looks like Kai got pretty far by being able to create a heapdump and isolating the problem to one specific file. Kai also mentioned that there was a lot of template code. I'm wondering whether BOOST was involved in Kai's case and if yes, what version of BOOST. Maybe if Kai could extract the list of include files pulled in in his case, then Sergey could reproduce the case with a minimal *.cpp file just pulling in those headers that are Open Source. In either case, generating a parser log file would help (both to identify the list of headers included, to get the statistics, and to see whether any syntax errors are involved). Please download attached indexer-debug-options.txt and save it in the root of your Eclipse install, then run Eclipse like this to pipe output into a file: cd eclipsehome java -XX:MaxPermSize=512m -Xmx2048m -jar plugins\org.eclipse.equinox.launcher_*.jar -debug indexer-debug-options.txt > indexer-debug.log now open your project in the UI and make sure that your problematic file gets indexed (for instance, right-click > Index > Generate Parser Log on the file).
I fear I am of not much help. Starting eclipse this time, with the given call, it get the heap dump while parsing another file (albeit of the same module). I append the parser log generated on this file. We are using boost, too, so this would be an option
Created attachment 229222 [details] indexer-debug-log
Created attachment 229223 [details] Parser Log
It would be really helpful if somebody could create a reproducible example using just the open source headers that requires more than 1GB to index.
It seems to be fixed for me on build id: 20130225-0426 (tested on debian sid x64 and debian wheezy x86)
Kai, would you be able to provide a similar heap dump from your workspace when indexing succeeds without running out of memory? Either using Indigo or a newer build (if you've found the problem has gone away as Javier suggests) would be equally helpful. I'm trying out a new kind of heap dump analysis tool and this problem is a good case study for it. Thanks very much, Robin
Where can I download a fixed build?
Below my experience with the problem, maybe can be helpful: I experienced the same problem for Juno (Eclipse IDE for C/C++ Developers / Version: Juno Service Release 2 / Build id: 20130225-0426) when trying to index a project that has boost dependency (quantlib.org). I did not try to modify the eclipse.ini, I jumped directly to nightly builds (Eclipse SDK /Version: 4.4.0/ Build id: N20130624-2000 and CDT from http://download.eclipse.org/tools/cdt/builds/kepler/nightly/). With the default eclipse.ini I got the same problem. But changing parameters to --launcher.XXMaxPermSize 1024m --launcher.defaultAction openFile --launcher.appendVmargs -vmargs -Xms256m -Xmx1024m The indexer worked fine. In a glance, an 'nightly' environment with more memory did not reproduce the problem.
(In reply to comment #33) > The indexer worked fine. > > In a glance, an 'nightly' environment with more memory did not reproduce the > problem. Indexing the same library (QuantLib), I got a ClassCastException, see bug 411650. It also ran out of memory with the default -Xmx512m, see screenshot.
Created attachment 232763 [details] Memory Analyzer screenshot
(In reply to comment #35) > Created attachment 232763 [details] > Memory Analyzer screenshot What are the titles of the columns in the Memory Analyzer screenshot?
Created attachment 232780 [details] Memory Analyzer screenshot, grouped by class Sorry, I was a by too aggressive with the cropping.
Created attachment 232781 [details] Memory Analyzer screenshot, no grouping
The HashMap that is shown in Memory Analyzer is most likely PDOM.fResultCache. Keys in that cache are AST bindings, so they hold the whole AST in memory. Maybe we are not calling PDOM.clearResultCache() as often as we should.
I don't think the PDOM.fResultCache map is the main cause of the memory issue Kai originally reported: it's only holding on to 50 MB of space in the attached heap dump. Might be an independent issue. I'm having a go at diagnosing this using a tool that can execute the CDT source code against the heap dump. I think I'm finding useful information but I don't know enough about the CDT implementation to arrive at any firm conclusions. I'm starting with the theory that there are a lot of duplicate CPPASTNames, where I'm defining "duplicate" as same name and node locations (i.e. using a unique key of "name.toString() + " - " + Arrays.toString(name.getNodeLocations())"). Does that definition make sense? Is there another source-level analysis I should try instead?
I believe I've made some real progress on this bug. Using the research tool I've developed that I mentioned earlier, I've been analyzing the set of CPPASTName instances by the source location they come from. There seems to be a lot of duplicates. That is, names with the same content and the same location, as calculated by methods like getNodeLocations(). I've attached some data files in case it helps find the actual bug. The first is the results of grouping the first 100,000 names by location. You can see just how many duplicates are coming from boost libraries, and especially macros in those libraries. The second is a list of the locations (more precisely this time - differentiating distinct expansions of the same macro) of all names for a particular symbol in the Boost library ("return_type_N_prot") - you can see that there are 15 instances for each occurrence of the macro in question (source here: http://www.boost.org/doc/libs/1_37_0/boost/lambda/detail/lambda_functor_base.hpp). I also attached the code I used. Perhaps we're ending up with a separate name instance every time the header is included? Please let me know if I've missed something, or if there's something else I can try running to further diagnose things. I could definitely use some guidance from someone who knows the code better than I do!
Created attachment 233395 [details] Duplicate names in the first 100,000 CPPASTName objects
Created attachment 233396 [details] Precise locations of all "return_type_N_prot" names
Created attachment 233397 [details] Duplicate CPPASTName analysis Java code
(In reply to comment #44) I't a common practice in boost library to include the same header multiple times but with different macro definitions so that each inclusion produces different code.
Ah, I'm not familiar with that technique. What would that looks like in source code? Would you have to manually undef the include guard in order to actually include the header text multiple times?
(In reply to comment #46) > Ah, I'm not familiar with that technique. What would that looks like in > source code? Would you have to manually undef the include guard in order to > actually include the header text multiple times? I'm not sure if multiple inclusion applies to lambda_functor_base.hpp. Usually it is done with files without include guards. You can enable /debug/scanner and /debug/scanner/missingIncludeGuards tracing options to get some insight into it.
Okay, I think I may have found the actual source of the leak. I discovered that each of the 15 duplicate "return_type_N_prot" names are actually contained by 15 separate parse trees for the same compilation unit. That is, 15 different instances of CPPASTTranslationUnit returning the same .cpp path from getFilePath(). Digging into what's keep the different trees alive, it turns out that one thread has references to all of them via thread locals. The "Worker-3" thread (0x1be3d568) retains 824MB of heap in its "threadLocals" field. It looks like the CPPClassSpecialization.fInProgress thread local is at least partially to blame for the leak. It's populated to avoid infinite recursion but never cleared. There may be other leaks caused by thread locals as well, as I haven't done the math to figure out if that one is responsible for all or most of the member, but it at least looks like a big chunk of it.
(In reply to comment #48) Thanks a lot for your investigative work. Submitted an attempted fix http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=f33ded195aa13f90b6c15abddaf1961eeb1da2a4.
*** cdt git genie on behalf of Sergey Prigogin *** Bug 400073 - Indexer runs out of memory. An attempt to fix a memory leak. [*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=f33ded195aa13f90b6c15abddaf1961eeb1da2a4
(In reply to comment #49) No problem! Honestly it was all for selfish reasons anyway. :) I needed a compelling case study for my research and this happened to fit the bill. > (In reply to comment #48) > Thanks a lot for your investigative work. Submitted an attempted fix > http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/ > ?id=f33ded195aa13f90b6c15abddaf1961eeb1da2a4.
(In reply to comment #51) > No problem! Honestly it was all for selfish reasons anyway. :) I needed a > compelling case study for my research and this happened to fit the bill. Pretty impressive. Will your research materialize into a new freeware or open source memory analysis tool?
I'm definitely planning on making the tool available, once I get past my paper deadline this week. :) It's currently just a set of plugins for the Eclipse Memory Analyzer tool, but I could make some features easier to use if I hacked the tool directly.
Robin, do you see any improvement with the latest changes?
You're asking the wrong person. :) I never actually reproduced the bug, or tired, really. I just worked with the heap dump that Kai uploaded.
(In reply to comment #49) > (In reply to comment #48) > Thanks a lot for your investigative work. Submitted an attempted fix > http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/ > ?id=f33ded195aa13f90b6c15abddaf1961eeb1da2a4. Any chance to get this into the 8.2.1 release?
(In reply to comment #56) > (In reply to comment #49) > > (In reply to comment #48) > > Thanks a lot for your investigative work. Submitted an attempted fix > > http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/ > > ?id=f33ded195aa13f90b6c15abddaf1961eeb1da2a4. > Any chance to get this into the 8.2.1 release? The change is in 8.2.1 already.
(In reply to comment #57) > (In reply to comment #56) > > (In reply to comment #49) > > > (In reply to comment #48) > > > Thanks a lot for your investigative work. Submitted an attempted fix > > > http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/ > > > ?id=f33ded195aa13f90b6c15abddaf1961eeb1da2a4. > > Any chance to get this into the 8.2.1 release? > > The change is in 8.2.1 already. Cool. The git genie script (comment #50 ) did not say anything about which branch. So I expected your commit would be in master only.
(In reply to Robin Salkeld from comment #53) > I'm definitely planning on making the tool available, once I get past my > paper deadline this week. :) It's currently just a set of plugins for the > Eclipse Memory Analyzer tool, but I could make some features easier to use > if I hacked the tool directly. Hi all, For anyone interested, I've made my tool available for installation into Eclipse. It adds a few additional custom queries to the Memory Analyzer tool. https://github.com/robinsalkeld/retrospect/ I've done my best to document the basics but it's still very much a research prototype. :) Cheers, Robin
I just updated to latest CDT 2.0.1.20130919 on Linux 64bit and the indexer still eats ups 16+GB RAM, standing at 4% steadily. We also have boost. :( Does that CDT version contain this fix mentioned to be in 8.2.1 SR?
(In reply to Mekk Elek from comment #60) > I just updated to latest CDT 2.0.1.20130919 on Linux 64bit and the indexer > still eats ups 16+GB RAM, standing at 4% steadily. What is 2.0.1.20130919? The latest release CDT version id 8.2.1.
(In reply to Sergey Prigogin from comment #61) > (In reply to Mekk Elek from comment #60) > > I just updated to latest CDT 2.0.1.20130919 on Linux 64bit and the indexer > > still eats ups 16+GB RAM, standing at 4% steadily. > > What is 2.0.1.20130919? The latest release CDT version id 8.2.1. It's the version for the C/C++ EPP (Eclipse IDE for C/C++ Developers).
(In reply to Marc-Andre Laperle from comment #62) > (In reply to Sergey Prigogin from comment #61) > > (In reply to Mekk Elek from comment #60) > > > I just updated to latest CDT 2.0.1.20130919 on Linux 64bit and the indexer > > > still eats ups 16+GB RAM, standing at 4% steadily. > > > > What is 2.0.1.20130919? The latest release CDT version id 8.2.1. > > It's the version for the C/C++ EPP (Eclipse IDE for C/C++ Developers). I just double checked and the EPP does contain the correct 8.2.1 version. C/C++ Development Tools 8.2.1.201309180223
I recently started using boost with Eclipse Kepler and I have run into memory issues as well. The CDT version in use is 8.2.1.201309180223. eclipse.ini looks like this: startup plugins/org.eclipse.equinox.launcher_1.3.0.v20130327-1440.jar --launcher.library plugins/org.eclipse.equinox.launcher.gtk.linux.x86_1.1.200.v20130807-1835 -product org.eclipse.epp.package.cpp.product --launcher.defaultAction openFile -showsplash org.eclipse.platform --launcher.XXMaxPermSize 1024m --launcher.defaultAction openFile --launcher.appendVmargs -vmargs -Dosgi.requiredJavaVersion=1.6 -Xms256m -Xmx1024m I have taken a heap dump using the Eclipse memory analyser and have run the Leak Suspects report. It finds two problem suspects. Out of a total heap of 603.4 MB suspect 1 uses 410 MB and suspect 2 uses 124 MB. Suspect 1: 29,584 instances of "java.lang.ref.Finalizer", loaded by "<system class loader>" occupy 429,936,896 (67.95%) bytes. Suspect 2: 8 instances of "org.eclipse.cdt.internal.core.parser.scanner.LocationMap", loaded by "org.eclipse.cdt.core" occupy 72,802,744 (11.51%) bytes. I have attached the dominator tree for one of the many Finalizer objects holding an org.eclipse.cdt.codan.core.cxx.model.CxxModelsCache object, which account for most of the memory use of the Finalizer. Maybe it is of help to someone. Best regards, Markus
Created attachment 238063 [details] Dominator tree for finalizer object
Fixed by commit http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=3fbe0d12af6ebf1a570defb83b105ba8d53ce59b. Everybody who experienced this problem, please verify that there is no remaining leak.
*** cdt git genie on behalf of Sergey Prigogin *** Bug 400073 - Indexer runs out of memory. Fixed a soft memory leak caused by accumulation of data in PDOM.fResultCache when several consecutive files fail to parse. [*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=3fbe0d12af6ebf1a570defb83b105ba8d53ce59b
*** cdt git genie on behalf of Sergey Prigogin *** Bug 400073 - Indexer runs out of memory. An attempt to reduce memory consumption. [*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=fb08c843e9ac5d03365a8407a8bf7e8671a2097b
(In reply to Sergey Prigogin from comment #66) > Everybody who experienced this problem, please verify that there is no > remaining leak. If you can point me to instructions on how to do this, I would gladly test your fix.
(In reply to Markus Schöpflin from comment #69) > (In reply to Sergey Prigogin from comment #66) > > Everybody who experienced this problem, please verify that there is no > > remaining leak. > > If you can point me to instructions on how to do this, I would gladly test > your fix. You can use this update site in "Install New Software" to update your CDT to the lastest nightly build: https://hudson.eclipse.org/cdt/job/cdt-master/lastSuccessfulBuild/artifact/releng/org.eclipse.cdt.repo/target/repository/
*** cdt git genie on behalf of Sergey Prigogin *** Bug 400073 - More robust clearing of results cache. [*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=ff4977523552c06f960ab9a124ff3c9f5bef7b7b
*** cdt git genie on behalf of Sergey Prigogin *** Bug 400073 - More robust clearing of results cache. [*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=63c70d737ea11a2ee6de1f4924c892ab72d6b3c0
*** cdt git genie on behalf of Sergey Prigogin *** Bug 400073 - Indexer runs out of memory. Fixed a soft memory leak caused by accumulation of data in PDOM.fResultCache when several consecutive files fail to parse. [*] http://git.eclipse.org/c/cdt/org.eclipse.cdt.git/commit/?id=d144de5c1d527ab402fcc38f915790df8b582ec3
I tested the fix using the CDT version 8.3.0.201312061958 and the results are impressive. Whilst the heap size does jump to the configured maximum value (990M in my case) from time to time, it always returns to the same level of about 250MB. I also observed a general speed improvement for suggestions and code completions; and the indexer now finds references which it did not find before. (Interestingly, the indexer seems unable to merge C and C++ references, as I'm getting two distinct sets of references when looking up certain symbols either from C or C++ code, but this is another matter.) Thank you very much for the fix!
Similar problem after just unzipped Kepler on a 64x Ubuntu 13.10 machine. Opened marketplace then got: ./eclipse 2014-02-21 16:19:45,920 [Worker-0] INFO o.e.m.c.i.i.nexus.NexusIndexManager - Updating index for repository: nexus|http://localhost:8081/nexus/content/groups/public 2014-02-21 16:19:45,962 [Worker-0] INFO c.n.h.c.p.n.NettyAsyncHttpProvider - Number of application's worked threads is 16 2014-02-21 16:19:46,033 [Worker-0] ERROR o.e.m.c.i.i.nexus.NexusIndexManager - Unable to update index for nexus|http://localhost:8081/nexus/content/groups/public java.io.IOException: Conexão recusada to http://localhost:8081/nexus/content/groups/public/.index/nexus-maven-repository-index.properties at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$PipedErrorInputStream.checkError(AsyncFetcher.java:250) ~[org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$PipedErrorInputStream.read(AsyncFetcher.java:258) ~[org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at java.io.PipedInputStream.read(PipedInputStream.java:378) ~[na:1.7.0_09] at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_09] at java.util.Properties$LineReader.readLine(Properties.java:434) ~[na:1.7.0_09] at java.util.Properties.load0(Properties.java:353) ~[na:1.7.0_09] at java.util.Properties.load(Properties.java:341) ~[na:1.7.0_09] at org.apache.maven.index.updater.DefaultIndexUpdater.downloadIndexProperties(DefaultIndexUpdater.java:457) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.access$100(DefaultIndexUpdater.java:75) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater$IndexAdaptor.setProperties(DefaultIndexUpdater.java:607) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:788) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:135) ~[indexer-core-3.1.0.jar:3.1.0] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateRemoteIndex(NexusIndexManager.java:1126) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateIndex(NexusIndexManager.java:1083) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager$1.run(NexusIndexManager.java:660) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.IndexUpdaterJob.run(IndexUpdaterJob.java:72) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53) [org.eclipse.core.jobs_3.5.300.v20130429-1813.jar:na] java.net.ConnectException: Conexão recusada to http://localhost:8081/nexus/content/groups/public/.index/nexus-maven-repository-index.properties at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95) ~[na:na] at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381) ~[na:na] at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372) ~[na:na] at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276) ~[na:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[na:na] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44) ~[na:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) ~[na:1.7.0_09] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) ~[na:1.7.0_09] at java.lang.Thread.run(Thread.java:722) ~[na:1.7.0_09] Caused by: java.net.ConnectException: Conexão recusada at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_09] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692) ~[na:1.7.0_09] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276) ~[na:na] ... 3 common frames omitted 2014-02-21 16:19:47,548 [Worker-6] WARN o.e.m.c.i.embedder.EclipseLogger - The artifact jdom:jdom:jar:1.1 has been relocated to org.jdom:jdom:jar:1.1 Exception in thread "ModalContext" Exception: java.lang.OutOfMemoryError thrown from the UncaughtExceptionHandler in thread "ModalContext" Error while logging event loop exception:
(In reply to Tesso Costa from comment #75) Is it with CDT 8.3?
(In reply to Sergey Prigogin from comment #76) > (In reply to Tesso Costa from comment #75) > Is it with CDT 8.3? Im using the contents of eclipse-jee-kepler-SR1-linux-gtk-x86_64.tar.gz Java(TM) SE Runtime Environment (build 1.7.0_09-b05) Java HotSpot(TM) 64-Bit Server VM (build 23.5-b02, mixed mode) Well, just found out that i cant even enter on install details on about screen! oh good lord! 2014-02-21 16:39:18,835 [Worker-3] INFO o.e.m.c.i.i.nexus.NexusIndexManager - Updating index for repository: nexus|http://localhost:8081/nexus/content/groups/public 2014-02-21 16:39:18,869 [Worker-3] INFO c.n.h.c.p.n.NettyAsyncHttpProvider - Number of application's worked threads is 16 2014-02-21 16:39:18,924 [Worker-3] ERROR o.e.m.c.i.i.nexus.NexusIndexManager - Unable to update index for nexus|http://localhost:8081/nexus/content/groups/public java.io.IOException: Conexão recusada to http://localhost:8081/nexus/content/groups/public/.index/nexus-maven-repository-index.properties at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$PipedErrorInputStream.checkError(AsyncFetcher.java:250) ~[org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$PipedErrorInputStream.read(AsyncFetcher.java:258) ~[org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at java.io.PipedInputStream.read(PipedInputStream.java:378) ~[na:1.7.0_09] at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_09] at java.util.Properties$LineReader.readLine(Properties.java:434) ~[na:1.7.0_09] at java.util.Properties.load0(Properties.java:353) ~[na:1.7.0_09] at java.util.Properties.load(Properties.java:341) ~[na:1.7.0_09] at org.apache.maven.index.updater.DefaultIndexUpdater.downloadIndexProperties(DefaultIndexUpdater.java:457) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.access$100(DefaultIndexUpdater.java:75) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater$IndexAdaptor.setProperties(DefaultIndexUpdater.java:607) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:788) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:135) ~[indexer-core-3.1.0.jar:3.1.0] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateRemoteIndex(NexusIndexManager.java:1126) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateIndex(NexusIndexManager.java:1083) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager$1.run(NexusIndexManager.java:660) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.IndexUpdaterJob.run(IndexUpdaterJob.java:72) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53) [org.eclipse.core.jobs_3.5.300.v20130429-1813.jar:na] java.net.ConnectException: Conexão recusada to http://localhost:8081/nexus/content/groups/public/.index/nexus-maven-repository-index.properties at com.ning.http.client.providers.netty.NettyConnectListener.operationComplete(NettyConnectListener.java:95) ~[na:na] at org.jboss.netty.channel.DefaultChannelFuture.notifyListener(DefaultChannelFuture.java:381) ~[na:na] at org.jboss.netty.channel.DefaultChannelFuture.notifyListeners(DefaultChannelFuture.java:372) ~[na:na] at org.jboss.netty.channel.DefaultChannelFuture.setFailure(DefaultChannelFuture.java:334) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:389) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276) ~[na:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[na:na] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44) ~[na:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) ~[na:1.7.0_09] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) ~[na:1.7.0_09] at java.lang.Thread.run(Thread.java:722) ~[na:1.7.0_09] Caused by: java.net.ConnectException: Conexão recusada at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) ~[na:1.7.0_09] at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:692) ~[na:1.7.0_09] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.connect(NioClientSocketPipelineSink.java:384) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.processSelectedKeys(NioClientSocketPipelineSink.java:354) ~[na:na] at org.jboss.netty.channel.socket.nio.NioClientSocketPipelineSink$Boss.run(NioClientSocketPipelineSink.java:276) ~[na:na] ... 3 common frames omitted 2014-02-21 16:39:20,256 [Worker-1] WARN o.e.m.c.i.embedder.EclipseLogger - The artifact jdom:jdom:jar:1.1 has been relocated to org.jdom:jdom:jar:1.1 Error while logging event loop exception: Error while logging event loop exception:
ok so my nexus repository is offline, but should not make eclipse crash right?
Ok, started nexus!! But still, permgen ./eclipse 2014-02-21 16:43:56,645 [Worker-0] INFO o.e.m.c.i.i.nexus.NexusIndexManager - Updating index for repository: nexus|http://localhost:8081/nexus/content/groups/public 2014-02-21 16:43:56,688 [Worker-0] INFO c.n.h.c.p.n.NettyAsyncHttpProvider - Number of application's worked threads is 16 2014-02-21 16:43:57,308 [Worker-4] WARN o.e.m.c.i.embedder.EclipseLogger - The artifact jdom:jdom:jar:1.1 has been relocated to org.jdom:jdom:jar:1.1 2014-02-21 16:43:57,627 [Worker-0] ERROR o.e.m.c.i.i.nexus.NexusIndexManager - Unable to update index for nexus|http://localhost:8081/nexus/content/groups/public java.io.IOException: Server returned status code 404: Not Found at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$PipedErrorInputStream.checkError(AsyncFetcher.java:250) ~[org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$PipedErrorInputStream.read(AsyncFetcher.java:258) ~[org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at java.io.PipedInputStream.read(PipedInputStream.java:378) ~[na:1.7.0_09] at java.io.InputStream.read(InputStream.java:101) ~[na:1.7.0_09] at java.util.Properties$LineReader.readLine(Properties.java:434) ~[na:1.7.0_09] at java.util.Properties.load0(Properties.java:353) ~[na:1.7.0_09] at java.util.Properties.load(Properties.java:341) ~[na:1.7.0_09] at org.apache.maven.index.updater.DefaultIndexUpdater.downloadIndexProperties(DefaultIndexUpdater.java:457) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.access$100(DefaultIndexUpdater.java:75) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater$IndexAdaptor.setProperties(DefaultIndexUpdater.java:607) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:788) ~[indexer-core-3.1.0.jar:3.1.0] at org.apache.maven.index.updater.DefaultIndexUpdater.fetchAndUpdateIndex(DefaultIndexUpdater.java:135) ~[indexer-core-3.1.0.jar:3.1.0] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateRemoteIndex(NexusIndexManager.java:1126) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager.updateIndex(NexusIndexManager.java:1083) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.NexusIndexManager$1.run(NexusIndexManager.java:660) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.m2e.core.internal.index.nexus.IndexUpdaterJob.run(IndexUpdaterJob.java:72) [org.eclipse.m2e.core_1.4.0.20130601-0317.jar:na] at org.eclipse.core.internal.jobs.Worker.run(Worker.java:53) [org.eclipse.core.jobs_3.5.300.v20130429-1813.jar:na] java.io.IOException: Server returned status code 404: Not Found at org.eclipse.m2e.core.internal.index.nexus.AsyncFetcher$MonitorListener.onStatus(AsyncFetcher.java:280) ~[na:na] at com.ning.http.client.SimpleAsyncHttpClient$BodyConsumerAsyncHandler.fireStatus(SimpleAsyncHttpClient.java:828) ~[na:na] at com.ning.http.client.SimpleAsyncHttpClient$BodyConsumerAsyncHandler.onStatusReceived(SimpleAsyncHttpClient.java:779) ~[na:na] at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.updateStatusAndInterrupt(NettyAsyncHttpProvider.java:1587) ~[na:na] at com.ning.http.client.providers.netty.NettyAsyncHttpProvider.messageReceived(NettyAsyncHttpProvider.java:1242) ~[na:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783) ~[na:na] at org.jboss.netty.handler.stream.ChunkedWriteHandler.handleUpstream(ChunkedWriteHandler.java:149) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783) ~[na:na] at org.jboss.netty.handler.codec.http.HttpContentDecoder.messageReceived(HttpContentDecoder.java:104) ~[na:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:783) ~[na:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:302) ~[na:na] at org.jboss.netty.handler.codec.replay.ReplayingDecoder.unfoldAndFireMessageReceived(ReplayingDecoder.java:522) ~[na:na] at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:506) ~[na:na] at org.jboss.netty.handler.codec.replay.ReplayingDecoder.messageReceived(ReplayingDecoder.java:443) ~[na:na] at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80) ~[na:na] at org.jboss.netty.handler.codec.http.HttpClientCodec.handleUpstream(HttpClientCodec.java:77) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564) ~[na:na] at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559) ~[na:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274) ~[na:na] at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261) ~[na:na] at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349) ~[na:na] at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:280) ~[na:na] at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200) ~[na:na] at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108) ~[na:na] at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44) ~[na:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) ~[na:1.7.0_09] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) ~[na:1.7.0_09] at java.lang.Thread.run(Thread.java:722) ~[na:1.7.0_09] Error while logging event loop exception: java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.defineClass(DefaultClassLoader.java:188) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClassHoldingLock(ClasspathManager.java:638) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClass(ClasspathManager.java:613) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findClassImpl(ClasspathManager.java:574) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClassImpl(ClasspathManager.java:492) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:465) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216) at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:395) at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:464) at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421) at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at org.eclipse.equinox.p2.ui.InstalledSoftwarePage.getProvisioningUI(InstalledSoftwarePage.java:259) at org.eclipse.equinox.p2.ui.InstalledSoftwarePage.createControl(InstalledSoftwarePage.java:71) at org.eclipse.ui.internal.about.InstallationDialog.tabSelected(InstallationDialog.java:274) at org.eclipse.ui.internal.about.InstallationDialog.createContents(InstallationDialog.java:245) at org.eclipse.jface.window.Window.create(Window.java:432) at org.eclipse.jface.dialogs.Dialog.create(Dialog.java:1104) at org.eclipse.jface.window.Window.open(Window.java:791) at org.eclipse.ui.internal.dialogs.AboutDialog$1.run(AboutDialog.java:127) at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70) at org.eclipse.ui.internal.dialogs.AboutDialog.buttonPressed(AboutDialog.java:122) at org.eclipse.jface.dialogs.Dialog$2.widgetSelected(Dialog.java:628) at org.eclipse.swt.widgets.TypedListener.handleEvent(TypedListener.java:248) at org.eclipse.swt.widgets.EventTable.sendEvent(EventTable.java:84) at org.eclipse.swt.widgets.Widget.sendEvent(Widget.java:1392) at org.eclipse.swt.widgets.Display.runDeferredEvents(Display.java:3742) at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3363) at org.eclipse.jface.window.Window.runEventLoop(Window.java:826) Logging exception: java.lang.OutOfMemoryError: PermGen space at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:791) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.defineClass(DefaultClassLoader.java:188) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClassHoldingLock(ClasspathManager.java:638) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.defineClass(ClasspathManager.java:613) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findClassImpl(ClasspathManager.java:574) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClassImpl(ClasspathManager.java:492) at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:465) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216) at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:395) at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:464) at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:421) at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:412) at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107) at java.lang.ClassLoader.loadClass(ClassLoader.java:356) at org.eclipse.ui.statushandlers.StatusManager.logError(StatusManager.java:277) at org.eclipse.ui.statushandlers.StatusManager.handle(StatusManager.java:201) at org.eclipse.ui.statushandlers.StatusManager.handle(StatusManager.java:231) at org.eclipse.ui.statushandlers.StatusManager.handle(StatusManager.java:242) at org.eclipse.ui.application.WorkbenchAdvisor.eventLoopException(WorkbenchAdvisor.java:326) at org.eclipse.ui.internal.ExceptionHandler.handleException(ExceptionHandler.java:65) at org.eclipse.jface.window.Window.runEventLoop(Window.java:830) at org.eclipse.jface.window.Window.open(Window.java:802) at org.eclipse.ui.internal.about.AboutHandler.execute(AboutHandler.java:32) at org.eclipse.ui.internal.handlers.HandlerProxy.execute(HandlerProxy.java:290) at org.eclipse.ui.internal.handlers.E4HandlerProxy.execute(E4HandlerProxy.java:90) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:601) at org.eclipse.e4.core.internal.di.MethodRequestor.execute(MethodRequestor.java:56) at org.eclipse.e4.core.internal.di.InjectorImpl.invokeUsingClass(InjectorImpl.java:243) java.lang.OutOfMemoryError: PermGen space Error while logging event loop exception: java.lang.OutOfMemoryError: PermGen space Logging exception: java.lang.OutOfMemoryError: PermGen space java.lang.OutOfMemoryError: PermGen space Error while logging event loop exception: java.lang.OutOfMemoryError: PermGen space Logging exception: java.lang.OutOfMemoryError: PermGen space
(In reply to Tesso Costa from comment #77) Kepler SR1 contains CDT 8.2.1, not 8.3.
Did this regress in CDT 8.5? I cannot get 32-bit Eclipse Luna SR1 to index Boost on Linux without running out of heap space, even when I give it a full 2GB to work with. I'm now trying 64-bit Eclipse Luna SR1 with CDT 8.5 and the following memory settings in eclipse.ini: -server -XX:MaxPermSize=320m -XX:+UseParallelOldGC -Xmx4g -Xmn1g It's currently taking a very long time at 26%, and has come close enough to the ~4GB limit a couple of times that it freezes up completely for a minute or two. I've disabled all plugins except for Codan and Devhelp.
(In reply to Benjamin Shadwick from comment #81) > Did this regress in CDT 8.5? I cannot get 32-bit Eclipse Luna SR1 to index > Boost on Linux without running out of heap space, even when I give it a full > 2GB to work with. It's definitely a regression, not sure when it was introduced. But yes, CDT runs out of memory indexing Boost, we've tried up to 8GB and still couldn't do it.
I've seen forum posts confirming this too. How can we get this bug report reopened?
(In reply to Benjamin Shadwick from comment #83) > How can we get this bug report reopened? Please create a new bug with detailed reproduction steps.
(In reply to Sergey Prigogin from comment #84) > (In reply to Benjamin Shadwick from comment #83) > > How can we get this bug report reopened? > > Please create a new bug with detailed reproduction steps. Create a project with the Boost source code. Boom. Not that complicated.
Update: Gave it 6GB of heap and it still eventually blew up. The interesting thing is that it does *not* blow up when I instead just add it as an exported preprocessor/macro include directory, even though I saw it subsequently index the boost directories for a few seconds. I guess it must ignore the "index unused headers" setting for headers that come in that way?
So while Doug claims "Any BOOST will blow up", it would still be good to get the _exact version_ for reproducing. Has this been posted anywhere ? In the past I have seen some versions of BOOST blowing up while other versions were fine. Could somebody perhaps attach the version of BOOST they are using ? And also let us know the host OS / host compiler version in use, since the host's gcc system headers may also influence the outcome.
(In reply to Martin Oberhuber from comment #87) And yes, one key difference between pulling in BOOST just thought the -I path versus having it in the project is that "in the project" forces CDT to index all files. Versus "pulled in by include" only those headers that are actually used are pulled in, AND they are pulled in with the correct context (a cpp file using them). That combined with the fact that I suppose few people on the earth understand BOOST well enough to actually edit / modify it, I would assume all of us just want to _USE_ it in order to get data structures properly indexed. Therefore I consider removing BOOST from the workspace (and just having it pulled in by -I path) more than just a workaround. It's a reasonable workflow IMO unless you deliberatly want to hurt yourself...
(In reply to Benjamin Shadwick from comment #86) > Update: Gave it 6GB of heap and it still eventually blew up. > > The interesting thing is that it does *not* blow up when I instead just add > it as an exported preprocessor/macro include directory, even though I saw it > subsequently index the boost directories for a few seconds. I guess it must > ignore the "index unused headers" setting for headers that come in that way? That's as expected. The problem is caused by one of the test files for boost, not the public headers. I've been able to easily reproduce this by creating a CDT project and importing the boost source, all of it, into it.
(In reply to Martin Oberhuber from comment #87) > Could somebody perhaps attach the version of BOOST they are using ? And also > let us know the host OS / host compiler version in use, since the host's gcc > system headers may also influence the outcome. Why not just try the latest one. I can't imagine how any recent version works. We've seen this for quite a while. I did it with mingw on Windows, but I don't think that matters. We've seen it with the QNX toolchain as well.
For me it was Boost 1.54 on Red Hat 6.2 x64. I won't be able to get the GCC version until Monday if it matters. While it's probably acceptable for most people's use to not directly index all of Boost, it seems to be a good test case for the indexer.
Any progress on this? I cant use eclipse Mars at all for our project at work, since the indexer will always crash. This obviously got way worse since Luna. I cant help out with sources, but it definitely is not limited to Boost. Our Project is ~370KLOCs, uses C++11 and the project includes multiple executables (which for example means multiple main functions in different directories). If the indexer just gracefully fails on a couple of these, or cant parse some complicated templates it wont be a problem (or would be easy to accept). Gobbing up all RAM and then failing is pretty serious
There appears to be a very sharp discontinuity in indexer behavior b/w Luna and Mars. I attempted to upgrade to Mars.1 today by copying the entire Luna workspace dir, deleting .metadata and .*project files, and recreating the same workspace and project settings. The upshot: Mars fails to complete a single index run with a -Xmx setting that's even twice as large as that used for Luna. Luna struggled but worked with -Xmx3g on ~1500 files. Mars, on the other hand, runs out of heap space before its Progress counter goes beyond "0/6xx sources, ...".
Hi everyone! I'd just like to ask if there's a plan to do something with this indexer problem? Does it lack human "resources", or additional information would be needed, or anything else is missing? I'm asking this because IMO it is a blocking issue for anyone who would like to use CDT for real-world C++ development: with the indexer working correctly, Eclipse+CDT is one of the best C++ IDEs currently available, but without proper indexing it's just a bad editor with a ridiculous amount of resource usage. To be honest, I'm a bit surprised that there is no automated (or at least manual) functional regression testing before each release, that would consist of importing a real-world, complicated, huge project (like boost, clang or something like that), and see if CDT can cope with it. By coping with it, I mean that some fixed set of indicators could be measured and compared to the previous versions (like the number of files processed, the number of symbols/references/whatever discovered in each file, the number of unresolved symbols, the duration and resource usage of the indexing, etc.). BTW, bug 441971 seems to be similar to this one.
For what it's worth, I've managed to avoid this issue for a while now by only adding my own project's source to my project tree, and bringing in dependencies only via header include paths. Even with this approach, Content Assist is too slow (which is a separate but probably related issue).
Yes, this bug renders eclipse unusable for our project, the Neon Release makes no difference. Just to restate the problem: after opening the project, eclipse will gobble up all heap space (I tried with up to 14Gb) and then report a crash in the indexer. Rinse and repeat. I understand that parsing C/C++ is complex, particularly if some headers/sources are mutually exclusive in different builds. But please, just add a recursion counter and bail out if it goes over 1000 (ideally give some reports with traces to help you find bugs or work around the problematic sources). I can live with some missing symbols, I cant live with constant crashes and disabling the indexer altogether ain`t a good option either
I am still using Luna SR2 to avoid this problem. To echo comment 95, it should be relatively simple to bisect the list of commits between Luna SR2 and the first version for which this bug was reopened. This is a blocking issue for Eclipse adoption by C++ community. "CDT does not work for C++ code" is a blocking issue. Hardly anything else can have a higher priority.
There's been a lot of discussion in this bug, so let me ask again to be sure: Is everyone who is experiencing this problem trying to index the *sources* (not just headers) of Boost, by creating a project for Boost itself (as opposed to just having Boost headers on the include path for other projects)? Or is just including Boost headers sufficient to cause this problem? Or are people seeing this in projects that don't use Boost at all? FWIW, I use CDT regularly for C++ development on projects that include many Boost headers, and I haven't seen this problem.
(In reply to Tamás Kiss from comment #95) > Hi everyone! I'd just like to ask if there's a plan to do something with > this indexer problem? Does it lack human "resources", or additional > information would be needed, or anything else is missing? Human resources is definitely a problem. There are only a handful of people maintaining the CDT indexer code, and many of us do so in our spare time only. Given that: (In reply to Igor Lubashev from comment #98) > To echo comment 95, it should be relatively simple to bisect the list of > commits between Luna SR2 and the first version for which this bug was > reopened. If someone who is actually suffering from this problem, could do such a bisection, that would be a huge help.
I am not using Boost at all, and am suffering from this issue https://bugs.eclipse.org/bugs/show_bug.cgi?id=400073#c92. Some points: *) The indexer always had crashes, just from Luna to Mars they got almost certain *) On Luna sometimes removing the .pdoms helps. *) I was unable to reduce the Project, it doesn`t seem to be entirely deterministic. *) The crashes leave little information on what happened. If I could re-construct the cause (sequence of parsed files, backtrace) I might be able to isolate the issue. *) I really doubt you will find *all* relevant bugs in a timely manner In my opinion, you should add a detection and limitation on recursion and/or heap-usage ASAP and allow us to provide you with reports on issues that seem to exists in various form on ALL eclipse version. Just expect that indexing might fail.
@(In reply to Nathan Ridge from comment #99) > Is everyone who is experiencing this problem trying to index the *sources* > (not just headers) of Boost, by creating a project for Boost itself (as > opposed to just having Boost headers on the include path for other projects)? I have a not too large project, which includes lots of Boost headers (including Boost.Test), OpenLDAP client library headers, and the Turtle Mock headers. The source code of those libraries are not imported. I haven't tried yet to import the project without the UT part, and drop the Boost.Test and Turtle Mock headers. Chances are that it might work, but I have to develop the UT code too, so it would not really be an solution.
Two of my customers have been in similar situations -- at some point, they wanted to import a large source code, and CDT took ages to index or ran OOME. How to identify the offending source code ? I propose that as a very first step, the CDT Indexer Tracing options should be improved to include timing information (what took how long to parse). That way, it's easier to find hot spots where the indexer is slow. This may also help identify problematic source code, which could then help creating reproducible test cases to improve the indexer. I've created https://bugs.eclipse.org/bugs/show_bug.cgi?id=497500 , could this simple enhancement be considered ?
As a suggestion to people having issues indexing large projects; I did much better when I indexed my large project as a bunch of Eclipse projects. It was a good fit for other reasons too, but indexing became more straightforward. Practically, if the indexer hangs or gets corrupted on an Eclipse project, you can reindex that project (deleting the .pdom file too if necessary), and this doesn't take too long so you can do it without wasting too much time. You have to specify inter-project dependencies this way but at least this is a fixed cost. I hate to be that guy, but maybe at this point in time, a serious consideration should be taking the time that would have gone towards fixing the old indexer and thinking about writing a new one that is clang based. The irony in my mind is that its problems notwithstanding, a few years ago, the CDT indexer was probably really the best multi platform option for a) code nav, b) auto complete, c) as-you-type error flagging w/ CODAN. Now, numerous projects are heavily exploiting clang. The result is indexing that is both more accurate and more reliable. If your projects compiles in clang, then it can be parsed by the clang frontend and indexed in a database, pure and simple. CODAN was a big plus of Eclipse a few years ago, now it's a weak point of the platform compared to projects like QtCreator or YouCompleteMe that offer as-you-type error flagging based on actual clang frontend parsing. This simply finds tons of errors that CODAN misses, and virtually never has false positives. This was one of the main motivators in my leaving Eclipse recently. I'd love to see Eclipse get a nice clang based indexer. The best part too is that by leaning heavily on clang, a ton of dev time that would go towards duplicating a C++ frontend, can instead go to more fruitful things, like trying to improve its performance or making sure the indexer is more transparent in its performance/correctness.
As a suggestion to people having issues indexing large projects; I did much better when I indexed my large project as a bunch of sub- Eclipse projects. It was a good fit for other reasons too, but indexing became more straightforward. Practically, if the indexer hangs or gets corrupted on an Eclipse project, you can reindex that project (deleting the .pdom file too if necessary), and this doesn't take too long so you can do it without wasting too much time. You have to specify inter-project dependencies this way but at least this is a fixed cost. I hate to be that guy, but maybe at this point in time, a serious consideration should be taking the time that would have gone towards fixing the old indexer and thinking about writing a new one that is clang based. The irony in my mind is that its problems notwithstanding, a few years ago, the CDT indexer was probably really the best multi platform option for a) code nav, b) auto complete, c) as-you-type error flagging w/ CODAN. Now, numerous projects are heavily exploiting clang. The result is indexing that is both more accurate and more reliable. If your projects compiles in clang, then it can be parsed by the clang frontend and indexed in a database, pure and simple. CODAN was a big plus of Eclipse a few years ago, now it's a weak point of the platform compared to projects like QtCreator or YouCompleteMe that offer as-you-type error flagging based on actual clang frontend parsing. This simply finds tons of errors that CODAN misses, and virtually never has false positives. This was one of the main motivators in my leaving Eclipse recently. I'd love to see Eclipse get a nice clang based indexer. The best part too is that by leaning heavily on clang, a ton of dev time that would go towards duplicating a C++ frontend, can instead go to more fruitful things, like trying to improve its performance or making sure the indexer is more transparent in its performance/correctness.
(In reply to Nir Friedman from comment #105) > I hate to be that guy, but maybe at this point in time, a serious > consideration should be taking the time that would have gone towards fixing > the old indexer and thinking about writing a new one that is clang based. It's OK to be that guy :). We've discussed this off and on over the last couple of years. However, the guy I want to hear from is the one who'll bring a team in to do the work. It would be a pretty massive undertaking to replace our current indexer and hook up all the features that depend on it it. Until then, we can only do what our small band of contributors can. It's cheaper at this point to fix these occasional bugs. BTW, we do have other bugs complaining specifically about Boost. Fix that and we probably have this one fixed too.
(In reply to Martin Oberhuber from comment #103) > Two of my customers have been in similar situations -- at some point, they > wanted to import a large source code, and CDT took ages to index or ran > OOME. How to identify the offending source code ? > > I propose that as a very first step, the CDT Indexer Tracing options should > be improved to include timing information (what took how long to parse). > That way, it's easier to find hot spots where the indexer is slow. This may > also help identify problematic source code, which could then help creating > reproducible test cases to improve the indexer. > > I've created https://bugs.eclipse.org/bugs/show_bug.cgi?id=497500 , could > this simple enhancement be considered ? Sounds good to me. Would you like to write the patch?
This bug was assigned and targeted at a now released milestone (or Future or Next that isn't being used by CDT). As that milestone has now passed, the milestone field has been cleared. If this bug has been fixed, please set the milestone to the version it was fixed in and mark the bug as resolved.
Steps to reproduce with publicly available source code on Windows 10 with Eclipse CDT 2021-03: you need git and python3 installed to complete those steps git clone https://github.com/prusa3d/Prusa-Firmware-Buddy.git cd Prusa-Firmware-Buddy git checkout b214307aeee53aa455441c77e1ed38f8415dfe84 cd utils python build.py --generate-cproject run eclipse CDT (I am using -vmargs -Xms1024m -Xmx12048m) install cmake4eclipse from eclipse marketplace (version 2.1.4 was used to reproduce) Use import existing project into workspace to import Prusa-Firmware-Buddy project. Project -> Build Configurations -> Set Active -> MINI_RELEASE_EMPTYBOOT In project properties -> C/C++ General -> Indexer Enable project specific settings, Store settings with project, Enable indexer, DO NOT Index source files not included in the build, DO NOT Index unused headers as C++ files, DO NOT Index unused headers as C files, Index all header variants, DO NOT Index source and header files opened in editor, Allow heuristic resolution of includes, Skip files larger than : 8 MB, Skip included files larger than: 16 MB, DO NOT Skip all references, DO NOT Skip implicit references, DO NOT Skip type and macro references, Use active build configuration, Reindex project on change of active build configuration Apply and Close Ctrl+B to build the project (so Cmake4eclipse gets compile time definitions) Right click project -> Index -> Rebuild (if the indexer is not running after the build) Eclipse runs out of memory after dozen of minutes when parsing the file /Buddy/src/gui/footer/ifooter_item.cpp
It can be also reproduced with CDT 10.3.1 in Eclipse 2021-06.
