Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: AW: [smila-dev] OutOfMemoryException during Crawl

Daniel,

How did you disable en-queueing of records and the BPEL processing? Did you modified Router rules, disabled it directly in the source code or somehow else?

Thanks,
Dmitry

Daniel.Stucky@xxxxxxxxxxx wrote:
Hi all,

Unfortunately I did not mention it, but I also used MemoryAnalyzer Tool :-)
In the test case we did not use all functionality of SMILA (we disabled the en-queueing of records and the BPEL processing of records) to only test behavior of Connectivity and Blackboard with XML and BinStore.
In this case 83% of memory was used by org.apache.commons.vfs.provider.local.LocalFile.


Bye,
Daniel



-----Ursprüngliche Nachricht-----
Von: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
bounces@xxxxxxxxxxx] Im Auftrag von Dmitry Hazin
Gesendet: Montag, 13. Oktober 2008 14:03
An: Smila project developer mailing list
Betreff: Re: [smila-dev] OutOfMemoryException during Crawl

Hi,

I found the tool that can be useful in detecting the leak source:
Memory
Analyzer tool (it's in incubation but seems to be working good),
http://www.eclipse.org/mat/
It allows to analyze Java heap dumps and to generate different reports
regarding memory usage. To get a dump on OOM Exception you should to
add
the following JVM parameter to SMILA.ini:
-XX:+HeapDumpOnOutOfMemoryError or -XX:+HeapDumpOnCtrlBreak to get heap
dump on demand.
Also you can get a dump with the following command: jmap
-dump:format=b,file=<filename.hprof> <pid> (command format id for
java6).

So here are the first results so far (leak analyzer executed on one
dump
was taken with filesystem crawler executed on large amount of files and
another when crawling job was stopped). The most memory is accumulated
by org.apache.ode and org.apache.commons.vfs, and org.apache.ode is
always at the first place:

Report with dump #1:

One instance of "org.apache.ode.bpel.engine.BpelServerImpl" loaded by
"org.apache.ode" occupies 55,544,144 (40.42%) bytes. The memory is
accumulated in one instance of
"org.apache.ode.bpel.memdao.BpelDAOConnectionImpl" loaded by
"org.apache.ode".
Keywords
org.apache.ode.bpel.engine.BpelServerImpl
org.apache.ode
org.apache.ode.bpel.memdao.BpelDAOConnectionImpl

Details »
Problem Suspect 2

20,031 instances of "org.apache.commons.vfs.provider.local.LocalFile",
loaded by "org.apache.commons.vfs" occupy 15,637,616 (11.38%) bytes.
These instances are referenced from one instance of
"java.util.HashMap$Entry[]", loaded by "<system class loader>"

Keywords
org.apache.commons.vfs
java.util.HashMap$Entry[]
org.apache.commons.vfs.provider.local.LocalFile


==============
Report with dump #2:

Problem Suspect 1

One instance of "org.apache.ode.bpel.engine.BpelServerImpl" loaded by
"org.apache.ode" occupies 43,411,080 (23.18%) bytes. The memory is
accumulated in one instance of
"org.apache.ode.bpel.memdao.BpelDAOConnectionImpl" loaded by
"org.apache.ode".
Keywords
org.apache.ode.bpel.engine.BpelServerImpl
org.apache.ode
org.apache.ode.bpel.memdao.BpelDAOConnectionImpl

Details »
Problem Suspect 2

The class "java.lang.ref.Finalizer", loaded by "<system class loader>",
occupies 39,970,744 (21.34%) bytes.
Keywords
java.lang.ref.Finalizer

Details »
Problem Suspect 3

33,379 instances of "org.apache.commons.vfs.provider.local.LocalFile",
loaded by "org.apache.commons.vfs" occupy 26,058,944 (13.92%) bytes.
These instances are referenced from one instance of
"java.util.HashMap$Entry[]", loaded by "<system class loader>"

Keywords
org.apache.commons.vfs
java.util.HashMap$Entry[]
org.apache.commons.vfs.provider.local.LocalFile


Thanks,
Dmitry

_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev


Back to the top