Community
Participate
Working Groups
Twice now with Oxygen M5, if I either move the search window or try to expand the search results while the search is ongoing, it freezes the application and I am left with killing eclipse.
After restarting this time I had an error dialog with java.lang.OutOfMemoryError: Java heap space that is a new installation with the default: -Xmx1024m
Please provide stack traces or steps.
The error log contains about 100 repeated messages about the OutOfMemoryError but the first few have a very short stacktrace. java.lang.OutOfMemoryError: Java heap space at org.eclipse.search.internal.core.text.FileCharSequenceProvider$Buffer.<init>(FileCharSequenceProvider.java:144) at org.eclipse.search.internal.core.text.FileCharSequenceProvider$FileCharSequence.findBufferToUse(FileCharSequenceProvider.java:358) at org.eclipse.search.internal.core.text.FileCharSequenceProvider$FileCharSequence.getBuffer(FileCharSequenceProvider.java:347) at org.eclipse.search.internal.core.text.FileCharSequenceProvider$FileCharSequence.length(FileCharSequenceProvider.java:326) at java.util.regex.Matcher.getTextLength(Unknown Source) at java.util.regex.Matcher.reset(Unknown Source) at java.util.regex.Matcher.reset(Unknown Source) As for the steps I was simply doing a file search on full workspace content (38k files). I have now updated my .ini to 4gb which is my normal setting and all is well, but there is still an issue as not everyone can allocate this much RAM.
Can you provide a test case or steps?
As I said do a text search with a large number of files on a warmed up workspaces (has done full built already and some call hierarchy search before). And make sure to constrain the max heap space. I can't think of anything else.
(In reply to Alain Picard from comment #5) > As I said do a text search with a large number of files What's "large" for you?
As stated in comment #3, it was 38k files
(In reply to Alain Picard from comment #7) > As stated in comment #3, it was 38k files Yes, I saw that. It's more about the matches. Can you try to figure out how many matches are reported before it gets an OOME?
In this case it wasn't that high, maybe 1k matches. One time about 1k and the other very few (< 100).
I tried to reproduce with -Xmx512M and searched 10'000 files finding over 1 million matches, and it did not run out of memory. Do you maybe have very large files (> 300 MB)?
Created attachment 267742 [details] Search freeze image Here is an image showing a new search that just froze at 0%. Eclipse was still responsive but it wouldn't complete saving the workspace on exit and I had to end the task. Nothing in the log and now I have 4gb for Xmx
(In reply to Dani Megert from comment #10) > I tried to reproduce with -Xmx512M and searched 10'000 files finding over 1 > million matches, and it did not run out of memory. > > Do you maybe have very large files (> 300 MB)? In terms of workspace files here are the biggest: 106M ./com.castortech.iris.models.ecore.ba/model2/ba2.aird 45M ./com.castortech.sharedjars/SourceZips/poi-src-3.9-20121203.zip 42M ./com.castortech.sharedjars/SourceZips/castor-1.3.3-RC1-src.zip
For some time around M5 I also had frequent search hangs when searching for text in all java and property files in my jdt workspace, but not in a reproducible way, so I never filed a bug. I haven't observed them in M6
(In reply to Till Brychcy from comment #13) > For some time around M5 I also had frequent search hangs when searching for > text in all java and property files in my jdt workspace, but not in a > reproducible way, so I never filed a bug. > I haven't observed them in M6 Alain, please try with 4.6 M6: https://www.eclipse.org/downloads/ or if you only need the Eclipse SDK: http://download.eclipse.org/eclipse/downloads/drops4/S-4.7M6-201703082000/
*** Bug 508447 has been marked as a duplicate of this bug. ***
Hi, I have encountered the same problem, except I have no OOME in the log file. Eclipse 4.7.0 (20170620-1800), Windows 10 What I do: 1/ Run file search over entire workspace (70k files, 139 matches) 2/ Click Expand all in the Search results view while the search is still running (some partial results are already present) 3/ Freeze - Application is not responding If I wait with the expand all until the search is finished no problem occurs.
Vikas, something for you to investigate?
I will investigate this for 4.8M1.
After couple of attempts on my normal workspace ( pde+platforui repos), searching for "pluginM" in the entire workspace and maximising/minimising search window causes comment#0. I also get !STACK 0 java.lang.OutOfMemoryError: Java heap space at org.eclipse.search.internal.core.text.FileCharSequenceProvider$Buffer.<init>(FileCharSequenceProvider.java:144) at org.eclipse.search.internal.core.text.FileCharSequenceProvider$FileCharSequence.findBufferToUse(FileCharSequenceProvider.java:358) at org.eclipse.search.internal.core.text.FileCharSequenceProvider$FileCharSequence.getBuffer(FileCharSequenceProvider.java:347) at org.eclipse.search.internal.core.text.FileCharSequenceProvider$FileCharSequence.length(FileCharSequenceProvider.java:326) at java.util.regex.Matcher.getTextLength(Unknown Source) at java.util.regex.Matcher.reset(Unknown Source) at java.util.regex.Matcher.reset(Unknown Source) at org.eclipse.search.internal.core.text.TextSearchVisitor.locateMatches(TextSearchVisitor.java:506) at org.eclipse.search.internal.core.text.TextSearchVisitor.access$8(TextSearchVisitor.java:504) at org.eclipse.search.internal.core.text.TextSearchVisitor$TextSearchJob.processFile(TextSearchVisitor.java:242) at org.eclipse.search.internal.core.text.TextSearchVisitor$TextSearchJob.run(TextSearchVisitor.java:185) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:56) !SUBENTRY 1 org.eclipse.core.jobs 4 2 2017-07-21 12:30:27.171 java.lang.OutOfMemoryError: Java heap space !SUBENTRY 1 org.eclipse.core.jobs 4 2 2017-07-21 12:30:27.171 !MESSAGE An internal error occurred during: "IDialogTestPass.java". !STACK 0 java.lang.OutOfMemoryError: GC overhead limit exceeded !SUBENTRY 1 org.eclipse.core.jobs 4 2 2017-07-21 12:30:27.171 !MESSAGE An internal error occurred during: "TestTableWrapLayout.java". !STACK 0 java.lang.OutOfMemoryError: GC overhead limit exceeded !SUBENTRY 1 org.eclipse.core.jobs 4 2 2017-07-21 12:30:27.171 !MESSAGE An internal error occurred during: "OverrideTestsSelectionProvider.java".
Created attachment 269488 [details] Heap size remaining decreases by 1MB with each call to constructor of Buffer
Since default eclipse.ini Xmx value is 1024MB and we know that each constructor call takes away 1MB ( see previous attachment), it would be best to reclaim memory on every 900th call of FileCharSequenceProvider.Buffer ( similar to org.eclipse.jdt.internal.core.index.Index.save() ). Please note that recreating this is slightly difficult but it "surely" happens if attempted for sometime. I will post predictable recreating steps as soon as I identify them.
Created attachment 269489 [details] I have all non-nested plugin of all these repos For me, this problem recreates "consistently" on Version: Photon (4.8) Build id: I20170714-2000 However as I said earlier, the steps to recreate this is not predictable but it surely happens ( within 4 attempts for me). I have all the non-nested projects of these repos ( see attachments) in a brand new workspace. I also have Preferences-> Plug-in Development-> "Include all plugins from target in Java search" option on.
I have the issue very often. May the new attachments will help to find the cause.
Created attachment 271251 [details] Dumped using jmap
Created attachment 271252 [details] Dumped using jstack
(In reply to Maik Greubel from comment #25) > Created attachment 271252 [details] > Dumped using jstack This indicates that org.eclipse.jst.jee.ui.internal.navigator.JEE5ContentProvider$1.run(JEE5ContentProvider.java:145) is doing work in the UI. Not sure which view this is. Can you try to find that view, close it and retry the search?
Seems like there is still too much investigation and work to do for it to fit in M4 deadline. Moving to M5.
+1 for this issue. It seems okay if you avoid interacting with the 'Search' view until it is finished searching. But the muscle memory from past Eclipse releases is strong - I tend to jump in as soon as I see the (partial) result I want.
Created attachment 272436 [details] another freeze
Created attachment 272437 [details] and another
Is there really nothing that you can do around this issue. I have seen very large group of people affected by this. It seems that all started when one "smart" eclipse developer decided that IO bound operation like file search, can be made faster when using ridiculous amount of worker threads. Eclipse was then unresponsive while doing file search. This was then "fixed" somehow by reducing the number of threads. But the file search is still freezing eclipse all the time. At 8%, 0% ... while consuming 100% of one core. A lot of times I am also getting out of memory during search (with -Xmx2048m). I have attached two thread dumps. Perhaps there is a problem, because workspace has the same file visible in multiple modules and they are dead locking themselves. I am using nested maven projects (with parent poms). The sad part is that I have to use the external tool doing file search in one thread and it is still faster and using less memory than eclipse every single time. It is file search ffs. How hard can it be to have it just working in 2018?
I've also encountered this issue on multiple occasions - I've tried waiting it out (5+ mins) to no effect. It happens regularly (not every search, but multiple times per week under normal "developer" usage), and the only recourse seems to be killing the process and restarting Eclipse. Still can't pinpoint what exactly triggers the freeze - I've got nothing that seems related in my error log, and doing the exact same search after a kill-restart usually just "works" - the most recent one was 307 matches across a workspace with 6 projects with ~130K files (`find . -type f | wc -l`) I'm running "Eclipse IDE for Java Developers" Oxygen.2 Release (4.7.2) on Ubuntu. eclipse.ini is launching with decent amount of heap space: -Xms4096m -Xmx4096m and java version "1.8.0_151"
I used Java Mission Control to do a trace for a minute while searching in a big workspace for the letter "a". org.eclipse.search.ui.text.AbstractTextSearchResult.getMatches seems to use 35% for the runtime. I see also more than 3 Millions entries of *.text.FileMatch in the heap. See screenshots.
Created attachment 272614 [details] Hot methods
Created attachment 272615 [details] Object statistics
(In reply to Lars Vogel from comment #35) > Created attachment 272615 [details] > Object statistics How much heap your VM had? 512 MB? FileMatch uses 132 MB = 30%? BTW here is no trace of SWT resources
Same here on Oxygen.2. This is _really_ annoying. Normally I would wait for the next version to get this hopefully fixed and keep calm. But since it happens everytime I search, I re-activated my account here to complain. If I do _anything_ (e.g. clicking another window in eclipse) while file search is in progress. It locks up. We have a large project with 60+ maven projects in a workspace. This is really no fun killing the process everytime. I would love to see that fixed in a next SR/Hotfix not just in the next Eclipse major version. My colleagues already joking about me being the only grandpa still using Eclipse and not IntelliJ :). Maybe it's time to take a look... System: MacOS 10.11.4 (15E65), Eclipse Oxygen.2, JDK 1.8.0_162.
Created attachment 273640 [details] Screenshot: OOM Just happened on Eclipse Photon
Created attachment 273641 [details] 5 thread dumps created with jstack
raising severity; changed title since happening also on Photon
Some observations: - ProgressMonitor (progressProvider of JobManager) holds hundreds of TextSearchJob instances in managedJobs - much memory is held by FileCharSequenceProvider$Buffer (63%), they are held by TextSearchJob#charsequenceForPreviousLocation
> - ProgressMonitor (progressProvider of JobManager) holds hundreds of > TextSearchJob instances in managedJobs Mistake: It is org.eclipse.ui.internal.progress.ProgressManager, not ProgressMonitor
New Gerrit change created: https://git.eclipse.org/r/121254
(In reply to Eclipse Genie from comment #43) > New Gerrit change created: https://git.eclipse.org/r/121254 Thanks Karsten, looks good to me. With this patch I can search for "a" in a workspace containing JDT UI and a few test projects -> Result ~ 6 000 000 hits. With this patch I receive OOM. Tested also with a few random strings and the result set is always the same, with patch of without.
> With this patch I receive OOM. > > Tested also with a few random strings and the result set is always the same, > with patch of without. I hope you meant "Without this patch" you receive OOM and with it not anymore.
(In reply to Lars Vogel from comment #44) > (In reply to Eclipse Genie from comment #43) > > New Gerrit change created: https://git.eclipse.org/r/121254 > > Thanks Karsten, looks good to me. With this patch I can search for "a" in a > workspace containing JDT UI and a few test projects -> Result ~ 6 000 000 > hits. > > With this patch I receive OOM. You probably mean *without* the patch. While I see that the patch is OK, I can't reproduce the problem on my machine, with or without the patch. Searching for "a" in eclipse.platform.releng.aggregator finds ~1.2 billion matches, but the memory consumption in both cases is nearly same, at 200-300 MB, with no OOM if I set Xmx to 1 GB. If I search for a|e regex, I get ~3.2 billions matches and memory use of ~500MB, however even here I see that the tree/table viewers have a problem to manage such high number of entries - and still no OOM. The UI surely freezes during this kind of search, and the freeze is worse for the "tree" layout of the search UI. So I'm not sure that the proposed change is the fix for the root cause (which is still not clear for me) - or I need a better steps to reproduce. (In reply to Karsten Thoms from comment #41) > Some observations: > - ProgressMonitor (progressProvider of JobManager) holds hundreds of > TextSearchJob instances in managedJobs This is fishy. How this can happen? The jobs should be removed after they are done.
> This is fishy. How this can happen? The jobs should be removed after they > are done. Agreed, this looked also strange to me. I don't know when exactly they are removed. Maybe we could create a follow-up bug to investigate this? The patch might not be fixing the root cause, but it definetely fixes a leak and will most certainly avoid the freeze. When the jobs would be removed and GC'ed, then those resources would also be freed automatically. I could provide a memory dump of the crash that I analyzed, but it is 2GB large.
Gerrit change https://git.eclipse.org/r/121254 was merged to [master]. Commit: http://git.eclipse.org/c/platform/eclipse.platform.text.git/commit/?id=34db1f81770632c6289bb024c2ab4d7284456a95
(In reply to Karsten Thoms from comment #45) > > With this patch I receive OOM. > > > > Tested also with a few random strings and the result set is always the same, > > with patch of without. > > I hope you meant "Without this patch" you receive OOM and with it not > anymore. Yes
(In reply to Karsten Thoms from comment #47) > > This is fishy. How this can happen? The jobs should be removed after they > > are done. > > Agreed, this looked also strange to me. I don't know when exactly they are > removed. > > Maybe we could create a follow-up bug to investigate this? Or just keep this one open? > The patch might > not be fixing the root cause, but it definetely fixes a leak and will most > certainly avoid the freeze. When the jobs would be removed and GC'ed, then > those resources would also be freed automatically. Sure, this is why I +2 on the patch. > I could provide a memory dump of the crash that I analyzed, but it is 2GB > large. Do you have steps to reproduce, or can you check in the debugger the number of the jobs in org.eclipse.ui.internal.progress.ProgressManager.managedJobs during the search? You probably can see in the dump number of instances of TextSearchJob? If the job count is huge, the fix would be to add a restriction to org.eclipse.search.internal.core.text.TextSearchVisitor.search(IFile[], IProgressMonitor).filesPerJob. Right now it is "(files.length + jobCount - 1) / jobCount" and jobCount is (files.length + 49) / 50. So if the job number the key, the OOM bug should appear faster with bigger amount of files in the search scope, not because of the number of found matches. High number of matches causes UI freeze due the tableviewer refreshes, but not necessarily OOM.
fFiles.length is 83830 The dump contains 550 TextSearchJob instances ProgressManager#managedJobs.size = 387 Sorry, I don't have steps to reproduce, it just happened while searching through my Xtext dev workspace. I think it is more due to large files in the search scope, and the files length of 83k is also high. I am not sure if we should track more in this bug, the freeze should be gone and we might not be able to fix the root for Photon. Maybe we leave it a short time open and we see if we could do more now.
(In reply to Andrey Loskutov from comment #50) > So if the job number the key, the OOM bug should appear faster with bigger > amount of files in the search scope, not because of the number of found > matches. High number of matches causes UI freeze due the tableviewer > refreshes, but not necessarily OOM. Yep. I've just created a dummy project in the root of my home/git with 369692 files / 6204 jobCount and a nice OOM while searching for "the text you will never find nowhere!!!". Reducing jobCount to 100 fixed OOM's. (In reply to Karsten Thoms from comment #51) > fFiles.length is 83830 > The dump contains 550 TextSearchJob instances > ProgressManager#managedJobs.size = 387 Matches my theory. > I am not sure if we should track more in this bug, the freeze should be gone > and we might not be able to fix the root for Photon. Maybe we leave it a > short time open and we see if we could do more now. I think the patch is trivial (now we know the root cause), I will push it in few minutes.
Perfect :)
New Gerrit change created: https://git.eclipse.org/r/121286
(In reply to Eclipse Genie from comment #54) > New Gerrit change created: https://git.eclipse.org/r/121286 To reproduce the bug, simply have a project with ~1000000 files and search for a string which does not exist - in a VM with Xmx1G Eclipse will fail with OOM pretty soon (without our patches). With my or Karsten patch there will be no OOM. This patch simply reduces the number of started TextSearchJob to a maximum of 100. This is still needed after the patch from Karsten, because next time someone adding another field to the TextSearchJob will introduce OOM again.
Gerrit change https://git.eclipse.org/r/121286 was merged to [master]. Commit: http://git.eclipse.org/c/platform/eclipse.platform.text.git/commit/?id=23730518262552f9936ef1545a7fa6e83855f154
I think we can close it now?
(In reply to Karsten Thoms from comment #57) > I think we can close it now? Sure. Thanks for the looking into this bug.
*** Bug 519185 has been marked as a duplicate of this bug. ***
Tested with two workspace which always would fails with heap errors during file search. Both work fine and super fast now. Thanks Karsten and Andrey.
*** Bug 533220 has been marked as a duplicate of this bug. ***
