[news.eclipse.tools.hyades] Re: Extreme performance degradation while pr

[curtispd@xxxxxxxxxx]

Hi Dave,

This is a good question. I'm guessing the reason for this extreme 
performance loss is because of the vast amounts of data that have to be 
collected and processed. Consider the case where you have many very small 
and fast methods that are called a very large number of times. In order to 
collect execution data, the data collector will send a record of every 
single method entry and exit, along with a timestamp that we use to 
calculate times.

I have a few suggestions for dealing with this:

1. The first one is easy and you suggested it; profile to a file. This 
will essentially redirect the raw data coming in to a file without 
processing it at all. The contents stored will be zip compressed; so you 
'll be able to judge by the size of this file how much data is coming in. 
Because the parsing of the data is much slower than the generating on the 
data collection side, it shouldn't take as long. Then you would import the 
file. Rather than letting it grind all night while you're asleep, you can 
tell it to import, say, the first quarter of the file. If the operation 
has different phases; you may be able to pick out each phase and look at 
them separately. Along the same lines, under the Profiling tab in the 
launch configuration dialog, there is a Limits tab. This will allow you to 
set a limit after which no more data will be collected, if you want to 
look at the first part of the operation. In addition, you can manually 
start and pause monitoring while the operation is running, say, if you 
want to start collecting at a certain point in the operation.

2. Narrow down the filtering to, say, filter out certain methods. If there 
is a small, fast method that is called millions of times, you can put in a 
filter on this specific method. Note, the first filter that matches will 
be applied, starting from the top. While you'll no longer get data about 
this method, you can still look at cumulative times of the caller to get 
meaningful times. Base time means time strictly inside the method, and 
cumulative also includes any method that are called within this method. In 
order to identify which methods might be culprits, you might want to 
profile the very start of the operation and looking at the method 
statistics; sort by one of the time columns.

Hope this helps.

Curt