Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
AW: [smila-dev] OutOfMemoryException during Crawl

Hi all,

Unfortunately I did not mention it, but I also used MemoryAnalyzer Tool :-)
In the test case we did not use all functionality of SMILA (we disabled the en-queueing of records and the BPEL processing of records) to only test behavior of Connectivity and Blackboard with XML and BinStore.
In this case 83% of memory was used by org.apache.commons.vfs.provider.local.LocalFile.


Bye,
Daniel



> -----Ursprüngliche Nachricht-----
> Von: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> bounces@xxxxxxxxxxx] Im Auftrag von Dmitry Hazin
> Gesendet: Montag, 13. Oktober 2008 14:03
> An: Smila project developer mailing list
> Betreff: Re: [smila-dev] OutOfMemoryException during Crawl
> 
> Hi,
> 
> I found the tool that can be useful in detecting the leak source:
> Memory
> Analyzer tool (it's in incubation but seems to be working good),
> http://www.eclipse.org/mat/
> It allows to analyze Java heap dumps and to generate different reports
> regarding memory usage. To get a dump on OOM Exception you should to
> add
> the following JVM parameter to SMILA.ini:
> -XX:+HeapDumpOnOutOfMemoryError or -XX:+HeapDumpOnCtrlBreak to get heap
> dump on demand.
> Also you can get a dump with the following command: jmap
> -dump:format=b,file=<filename.hprof> <pid> (command format id for
> java6).
> 
> So here are the first results so far (leak analyzer executed on one
> dump
> was taken with filesystem crawler executed on large amount of files and
> another when crawling job was stopped). The most memory is accumulated
> by org.apache.ode and org.apache.commons.vfs, and org.apache.ode is
> always at the first place:
> 
> Report with dump #1:
> 
> One instance of "org.apache.ode.bpel.engine.BpelServerImpl" loaded by
> "org.apache.ode" occupies 55,544,144 (40.42%) bytes. The memory is
> accumulated in one instance of
> "org.apache.ode.bpel.memdao.BpelDAOConnectionImpl" loaded by
> "org.apache.ode".
> Keywords
> org.apache.ode.bpel.engine.BpelServerImpl
> org.apache.ode
> org.apache.ode.bpel.memdao.BpelDAOConnectionImpl
> 
> Details »
> Problem Suspect 2
> 
> 20,031 instances of "org.apache.commons.vfs.provider.local.LocalFile",
> loaded by "org.apache.commons.vfs" occupy 15,637,616 (11.38%) bytes.
> These instances are referenced from one instance of
> "java.util.HashMap$Entry[]", loaded by "<system class loader>"
> 
> Keywords
> org.apache.commons.vfs
> java.util.HashMap$Entry[]
> org.apache.commons.vfs.provider.local.LocalFile
> 
> 
> ==============
> Report with dump #2:
> 
> Problem Suspect 1
> 
> One instance of "org.apache.ode.bpel.engine.BpelServerImpl" loaded by
> "org.apache.ode" occupies 43,411,080 (23.18%) bytes. The memory is
> accumulated in one instance of
> "org.apache.ode.bpel.memdao.BpelDAOConnectionImpl" loaded by
> "org.apache.ode".
> Keywords
> org.apache.ode.bpel.engine.BpelServerImpl
> org.apache.ode
> org.apache.ode.bpel.memdao.BpelDAOConnectionImpl
> 
> Details »
> Problem Suspect 2
> 
> The class "java.lang.ref.Finalizer", loaded by "<system class loader>",
> occupies 39,970,744 (21.34%) bytes.
> Keywords
> java.lang.ref.Finalizer
> 
> Details »
> Problem Suspect 3
> 
> 33,379 instances of "org.apache.commons.vfs.provider.local.LocalFile",
> loaded by "org.apache.commons.vfs" occupy 26,058,944 (13.92%) bytes.
> These instances are referenced from one instance of
> "java.util.HashMap$Entry[]", loaded by "<system class loader>"
> 
> Keywords
> org.apache.commons.vfs
> java.util.HashMap$Entry[]
> org.apache.commons.vfs.provider.local.LocalFile
> 
> 
> Thanks,
> Dmitry
> 
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev


Back to the top