[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
RE: [platform-core-dev] RE: core.resources Performance Test question

I've opened bug 294043 to investigate tests for temporary memory consumption.

Martin, Boris, I've already CC'ed you. Other interested are welcome.

Pawel


From: "Oberhuber, Martin" <Martin.Oberhuber@xxxxxxxxxxxxx>
To: "Eclipse Platform Core component developers list." <platform-core-dev@xxxxxxxxxxx>
Cc: Frederic Fusier <frederic_fusier@xxxxxxxxxx>, John Arthorne <John_Arthorne@xxxxxxxxxx>, Boris Bokowski <Boris_Bokowski@xxxxxxxxxx>
Date: 10/28/2009 12:13 PM
Subject: RE: [platform-core-dev] RE: core.resources Performance Test question
Sent by: platform-core-dev-bounces@xxxxxxxxxxx




Hehe,

I know about binary search. What I was trying to say is, if the normal
test case has "just enough"
Memory it makes no sense to try with half the memory since that is bound
to fail. Also, it is
Probably not relevant to know how much memory it needs exactly. It may
be sufficient to know
That it does need less memory that before.

But yes, if the goal is finding out the exact amount of memory needed, a
binary search makes sense.

:)
Martin

-----Original Message-----
From: platform-core-dev-bounces@xxxxxxxxxxx
[mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of Boris
Bokowski
Sent: Wednesday, October 28, 2009 11:53 AM
To: Eclipse Platform Core component developers list.
Cc: Frederic Fusier; John Arthorne; Boris Bokowski
Subject: Re: [platform-core-dev] RE: core.resources Performance Test
question

What I meant with "binary search" was that you would use a strategy to
find the memory requirements using a minimal number of "experiments".
Each experiment should reduce the interval in which the end result lies
by half.

E.g. If you start with 256, the next try would be with 128. If 128 is
not enough, you would next try with (256+128)/2 and so on.

Boris

On 2009-10-27, at 22:56, "Oberhuber, Martin"
<Martin.Oberhuber@xxxxxxxxxxxxx  > wrote:

> Hi Boris,
>
> Actually I do not think a binary search would be needed.
>
> Assuming that the original test is written such that it "just works"
> with
> The default heap size, there are two options for checking "how much
> less"
> Memory the test could handle:
>
>  (a) Increase the amount of work that the test does, but keep the heap
>      size unchanged. E.g. perform the test with 5000 files, 6000
> files, ...
>
>  (b) Keep the amount of work unchanged but reduce heap size. E.g.
> start
>      with a default 256m, then 220m, 200m, ...
>
> I personally find option (b) more intersting since in option (a) the
> Complexity of the algorithm plays an important role ... An O(n)
> Algorithm would return totally different results than an O(n^2) One.
> Also, for end users it seems more interesting how much memory A given
> amount of work minimally needs.
>
> Cheers
> Martin
>
>
> -----Original Message-----
> From: platform-core-dev-bounces@xxxxxxxxxxx
> [mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of Boris
> Bokowski
> Sent: Saturday, October 17, 2009 3:29 AM
> To: Eclipse Platform Core component developers list.
> Cc: Frederic Fusier; John Arthorne; Boris Bokowski
> Subject: Re: [platform-core-dev] RE: core.resources Performance Test
> question
>
> Binary search for required heap size? Interesting idea, I think it's
> worth a try. Though I wonder how stable the results would be...
>
> Boris
>
>
>
> On 2009-10-16, at 17:44, "Oberhuber, Martin"
> <Martin.Oberhuber@xxxxxxxxxxxxx  > wrote:
>
>> Hi again,
>>
>> I thought a bit more about this, and I am afraid that the way the
>> test
>
>> (WorkspacePerformanceTest#testRefreshProject())
>> is written today, the heap size data will not at all help me.
>>
>> Right now, heap size is measured after the test completes; but in my
>> case (Refresh resources), memory consumption is temporary during the
>> algorithm. In other words, the algorithm needs a lot of memory, but
>> once the computation is done the memory is returned for garbage
>> collection so the value after the test is not interesting.
>>
>> I assume that there are many other similar cases, so perhaps somebody

>> has an idea for how to measure memory consumption in this situation?
>>
>> Right now, my feeling ist that the only reliable and realistic way to

>> measure memory consumption is running the test in a separate VM
>> process with varying memory limits set by -Xmx vm argument, until the

>> test runs into an OutOfMemoryException.
>> This seems to be the only way how garbage collection can be taken
>> into
>
>> account, coming up with a value that matters for end users.
>>
>> Has anybody tried this before? Or how else could I get realistic data

>> for memory consumption?
>>
>>
>> Cheers,
>> --
>> Martin Oberhuber, Senior Member of Technical Staff, Wind River Target

>> Management Project Lead, DSDP PMC Member
>> http://www.eclipse.org/dsdp/tm
>>
>>
>>
>>> -----Original Message-----
>>> From: platform-core-dev-bounces@xxxxxxxxxxx
>>> [mailto:platform-core-dev-bounces@xxxxxxxxxxx] On Behalf Of
>>> Oberhuber, Martin
>>> Sent: Friday, October 16, 2009 5:33 PM
>>> To: Frederic Fusier
>>> Cc: Eclipse Platform Core component developers list.; John Arthorne;

>>> Boris Bokowski
>>> Subject: [platform-core-dev] RE: core.resources Performance Test
>>> question
>>>
>>> Hi Frederic,
>>>
>>>> BTW, I had a look on my tool and saw that it was not too costly to
>>>> finalize the implementation to display a new number
>>>
>>> that is great! - But I don't think I'll need the pages for the
>>> latest
>
>>> I-build since I'll want to compare an upcoming N-build or I-build on

>>> the official performance machines against the baseline anyways. So
>>> it'll be sufficient if that upcoming N-build or I-build has the new
>>> number (heapsize) displayed for both the new build and the previous
>>> baseline.
>>>
>>>> Would it be possible to open a bug for this against Platform/Releng

>>>> in order to track the work around this?
>>>
>>> You are welcome:
>>> https://bugs.eclipse.org/bugs/show_bug.cgi?id=292545
>>>
>>> Cheers,
>>> --
>>> Martin Oberhuber, Senior Member of Technical Staff, Wind River
>>> Target
>
>>> Management Project Lead, DSDP PMC Member
>>> http://www.eclipse.org/dsdp/tm
>> _______________________________________________
>> platform-core-dev mailing list
>> platform-core-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/platform-core-dev
> _______________________________________________
> platform-core-dev mailing list
> platform-core-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/platform-core-dev
> _______________________________________________
> platform-core-dev mailing list
> platform-core-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/platform-core-dev
_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev
_______________________________________________
platform-core-dev mailing list
platform-core-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/platform-core-dev