mat-dev http://dev.eclipse.org/mhonarc/lists/mat-dev/maillist.html mat-dev en-us Wed, 09 May 2012 10:00:04 GMT Wed, 09 May 2012 10:00:04 GMT http://blogs.law.harvard.edu/tech/rss MHonArc RSS 2.0 RCFile webmaster@eclipse.org (Webmaster) webmaster@eclipse.org (Webmaster) mat-dev http://www.eclipse.org/eclipse.org-common/themes/Phoenix/images/eclipse_home_header.jpg http://dev.eclipse.org/mhonarc/lists/mat-dev/maillist.html [mat-dev] Contribution to Juno M7 http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00332.html Just a short info – I used the build from yesterday (SVN revision 1342) as MAT for M7 of the Juno release.I re-enabled MAT which was temporary excluded from the simultaneous release build because of this: http://dev.eclipse.org/mhonarc/lists/cross-project-...

Just a short info – I used the build from yesterday (SVN revision 1342) as MAT for M7 of the Juno release.

I re-enabled MAT which was temporary excluded from the simultaneous release build because of this:

http://dev.eclipse.org/mhonarc/lists/cross-project-issues-dev/msg07545.html

 

Krum

 

 

]]> Wed, 09 May 2012 09:52:51 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00332.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum) Re: [mat-dev] Failing Hudson jon http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00331.html The connection problem is fixed now. Krum From: mat-dev-bounces@xxxxxxxxxxx [mailto:mat-dev-bounces@xxxxxxxxxxx] On Behalf Of Tsvetkov, KrumSent: Mittwoch, 2. Mai 2012 11:55To: Memory Analyzer Dev listSubject: [mat-dev] Failing Hudson jon Hi, just a short ...

The connection problem is fixed now.

 

Krum

 

From: mat-dev-bounces@xxxxxxxxxxx [mailto:mat-dev-bounces@xxxxxxxxxxx] On Behalf Of Tsvetkov, Krum
Sent: Mittwoch, 2. Mai 2012 11:55
To: Memory Analyzer Dev list
Subject: [mat-dev] Failing Hudson jon

 

Hi,

 

just a short info – in the last days the MAT Hudson job is failing because of connection timeout trying to download dtfj.

I’ve opened https://bugs.eclipse.org/bugs/show_bug.cgi?id=378222 for this.

 

Krum

 

]]> Fri, 04 May 2012 07:27:35 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00331.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum) [mat-dev] Failing Hudson jon http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00330.html Hi, just a short info – in the last days the MAT Hudson job is failing because of connection timeout trying to download dtfj. I’ve opened https://bugs.eclipse.org/bugs/show_bug.cgi?id=378222 for this. Krum

Hi,

 

just a short info – in the last days the MAT Hudson job is failing because of connection timeout trying to download dtfj.

I’ve opened https://bugs.eclipse.org/bugs/show_bug.cgi?id=378222 for this.

 

Krum

 

]]> Wed, 02 May 2012 09:55:32 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00330.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum) Re: [mat-dev] FW: [tools-pmc] Migrating from CVS to Git http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00329.html There's a page about migrating to git which you probably already know at http://wiki.eclipse.org/Git/Migrating_to_Git . Also, Chris Aniszczyk has offered his help on moving to Git in the past. If he still wants to help, it may be a good idea to ask him to ... Hi,
On 22.03.2012 08:51, Tsvetkov, Krum wrote: 
Not quite sure how long “near term” is, but I think we have to finally
start planning our move to Git.

There's a page about migrating to git which you probably already know at http://wiki.eclipse.org/Git/Migrating_to_Git . Also, Chris Aniszczyk has offered his help on moving to Git in the past. If he still wants to help, it may be a good idea to ask him to relate some hints and/or best practices for a move.


Kind regards,

Erik Brangs

]]> Thu, 22 Mar 2012 08:56:09 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00329.html erik.brangs@xxxxxxx (Erik Brangs) [mat-dev] FW: [tools-pmc] Migrating from CVS to Git http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00328.html FYI > We will be deprecating SVN in the near term; these projects should start thinking about migrating soon Not quite sure how long “near term” is, but I think we have to finally start planning our move to Git.I would suggest to wait until Juno (and MAT ...

FYI

 

> We will be deprecating SVN in the near term; these projects should start thinking about migrating soon

 

Not quite sure how long “near term” is, but I think we have to finally start planning our move to Git.

I would suggest to wait until Juno (and MAT 1.2) is released, and then figure out what would be the best way to move.

 

Krum

 

From: tools-pmc-bounces@xxxxxxxxxxx [mailto:tools-pmc-bounces@xxxxxxxxxxx] On Behalf Of Wayne Beaton
Sent: Mittwoch, 21. März 2012 19:28
To: Tools PMC mailing list
Subject: [tools-pmc] Migrating from CVS to Git

 

Greetings Tools PMC.

As you should hopefully already be aware, the Eclipse Foundation has declared a shutdown date for CVS of December 21/2012.

Several Tools projects are still using CVS. The list of projects, generated based on information provided in project metadata via the developer portal, is available via the project status page.

http://eclipse.org/projects/tools/status.php?top=tools&cvs=1

Note that this link shows a filtered view of Tools projects that have a CVS repository.

Can you please ensure that the projects listed on this page have a plan for migration before the cut off date?

Also note that the status page includes a liveliness indicator. Some of these projects may be candidates for termination or consolidation.

Note yet some more that several projects currently have SVN. We will be deprecating SVN in the near term; these projects should start thinking about migrating soon.

http://eclipse.org/projects/tools/status.php?top=tools&svn=1

Let me know if I can help.

Wayne

--
Wayne Beaton
The Eclipse Foundation
Twitter: @waynebeaton
EclipseCon 2012AGILEALM 2012

PNG image

GIF image

_______________________________________________
tools-pmc mailing list
tools-pmc@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/tools-pmc
]]> Thu, 22 Mar 2012 07:51:26 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00328.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum) [mat-dev] Juno M6 contribution http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00327.html Just a short info – I contributed the MAT build results of revision 1301 to Juno M6. The milestone should be finalized by the end of the week. Krum

Just a short info – I contributed the MAT build results of revision 1301 to Juno M6. The  milestone should be finalized by the end of the week.

 

Krum

]]> Wed, 21 Mar 2012 14:24:37 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00327.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum) Re: [mat-dev] Huge heaps http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00326.html One other idea would be that MAT could, if possible, try to break up a dump into multiple "virtual" dumps (if there are large enough mutually exclusive subsets of the object graph) that are loaded as separate tabs and ISnapshots, but somehow still refer t... One other idea would be that MAT could, if possible, try to break up a dump into multiple "virtual" dumps (if there are large enough mutually exclusive subsets of the object graph) that are loaded as separate tabs and ISnapshots, but somehow still refer to each other indirectly (e.g. for the class definitions and boot classloader classes). This would also be separately interesting because I've always wanted a way to take a retained set and get the full power of MAT's ISnapshot analysis on it, instead of just a histogram and the limited investigation from there.

Also, what about having a separate build of MAT where ints are converted to longs (and alternative data structures such as BitFields would have to support longs)? This could perhaps be done by annotating ints that can be converted and creating a program that creates a copied code base with longs. The int version would be the recommended & primary version of MAT, but very advanced and large customers who have the extra memory and CPU to deal with the additional overhead of referring to longs could use this separate build. Of course this might also break any extensions that use object IDs and those would need separate builds too.

This would actually be a good case where C's #define directives would be useful to more easily create separate builds :)

--
Kevin Grigorenko
IBM WAS SWAT Team
kevin.grigorenko@xxxxxxxxxx
Blog: https://www.ibm.com/developerworks/mydeveloperworks/blogs/kevgrig/


Inactive hide details for "Tsvetkov, Krum" ---03/14/2012 09:10:42 AM---Andrew, the biggest dump I've worked with was about 8Gb "Tsvetkov, Krum" ---03/14/2012 09:10:42 AM---Andrew, the biggest dump I've worked with was about 8Gb and contained about 190.000.000 objects. I d


From:
"Tsvetkov, Krum" <krum.tsvetkov@xxxxxxx>

To:
Memory Analyzer Dev list <mat-dev@xxxxxxxxxxx>,

Date:
03/14/2012 09:10 AM

Subject:
Re: [mat-dev] Huge heaps

Sent by:
mat-dev-bounces@xxxxxxxxxxx



Andrew, the biggest dump I've worked with was about 8Gb and contained about 190.000.000 objects. I don't know what the number of references was.

I don't know how often it happens that people need to analyze dumps with more than 2^31 refs.
Although it doesn't sound good that MAT has some limitations in this area, I think it will be pretty challenging to break this limit.

You already listed some of the problems which may occur by defining the Id as unsigned.
I believe there are even more, and probably some of them are difficult to spot. For example, as far as I remember our own implementation of BitField can only work properly with positive numbers.

My personal opinion is that for the moment we should rather accept the limitation and document it, rather than trying to fix it for Juno.
And in general, if we try to fix it for a later release, I would suggest that we keep an eye on the performance and our own memory usage, and try not introduce serious degradation there (because of moving away from simple int[] or using some bigger storage than an int). I believe that the dumps which have over 2^31 refs are still rather something exceptional, and would like to keep the rest (probably 99%) of the users happy :-)

What do you think?

What is the experience of others so far?

Krum

-----Original Message-----
From: mat-dev-bounces@xxxxxxxxxxx [mailto:mat-dev-bounces@xxxxxxxxxxx] On Behalf Of Andrew Johnson
Sent: Dienstag, 13. März 2012 11:48
To: Memory Analyzer Dev list
Subject: [mat-dev] Huge heaps

I'm looking at bug 372548: ArrayIndexOutOfBoundsException with huge dump
https://bugs.eclipse.org/bugs/show_bug.cgi?id=372548

I'm working on a fix to one problem - the 1 to N indexes didn't cope with
more than 2^31 outbound references in the whole snapshot. It used an int
returned from the header index to index into the body index of all the
outbound references. I hope to be able to commit a fix shortly.

What are the biggest heaps we need to deal with, in terms of objects or
total outbound references?

What other restrictions are there for large dumps?
Do we need a LongIndex1N which can have more than 2^31 outbounds in total?
Do we need more than 2^31 objects? Currently object id < 2^31, i.e. signed
int
We could defined object id as being unsigned. Possible problems include:
Identifier.reverse - a negative number is returned for not found
inbounds - where we temporarily encode some refs as negative
int SnapshotInfo.getNumberOfObjects()
int IClass.getNumberOfObjects()
int IClass.getObjectIds()
int [] Snapshot.getOutboundReferentIds()
SetInt can't hold enough ints
int [] Snapshot.getOutboundReferentIds(int[] objectIds, IProgressListener
progressListener) - can't return more than 2^31 items
int [] Snapshot.getInboundReferentIds(int[] objectIds, IProgressListener
progressListener) - can't return more than 2^31 items
Do we need to expose a IntIndexReader which can be indexed by unsigned int
/ longs for > 2^31 entries?
Do we need to make the InboundWriter work with huge dumps. It splits the
refs into separate log files, but can the contents of the log files get
too big to sort as int arrays?
Can we save memory on building indices, doing the GC, rebuilding indices,
calculating dominator tree?

Andrew Johnson






Unless stated otherwise above:
IBM United Kingdom Limited - Registered in England and Wales with number
741598.
Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU






_______________________________________________
mat-dev mailing list
mat-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/mat-dev
_______________________________________________
mat-dev mailing list
mat-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/mat-dev



GIF image

GIF image

]]> Wed, 14 Mar 2012 16:30:14 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00326.html kevin.grigorenko@xxxxxxx (Kevin Grigorenko) Re: [mat-dev] Huge heaps http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00325.html Andrew, the biggest dump I've worked with was about 8Gb and contained about 190.000.000 objects. I don't know what the number of references was. I don't know how often it happens that people need to analyze dumps with more than 2^31 refs. Although it doesn... Andrew, the biggest dump I've worked with was about 8Gb and contained about 190.000.000 objects. I don't know what the number of references was. I don't know how often it happens that people need to analyze dumps with more than 2^31 refs. Although it doesn't sound good that MAT has some limitations in this area, I think it will be pretty challenging to break this limit. You already listed some of the problems which may occur by defining the Id as unsigned. I believe there are even more, and probably some of them are difficult to spot. For example, as far as I remember our own implementation of BitField can only work properly with positive numbers. My personal opinion is that for the moment we should rather accept the limitation and document it, rather than trying to fix it for Juno. And in general, if we try to fix it for a later release, I would suggest that we keep an eye on the performance and our own memory usage, and try not introduce serious degradation there (because of moving away from simple int[] or using some bigger storage than an int). I believe that the dumps which have over 2^31 refs are still rather something exceptional, and would like to keep the rest (probably 99%) of the users happy :-) What do you think? What is the experience of others so far? Krum -----Original Message----- From: mat-dev-bounces@xxxxxxxxxxx [mailto:mat-dev-bounces@xxxxxxxxxxx] On Behalf Of Andrew Johnson Sent: Dienstag, 13. März 2012 11:48 To: Memory Analyzer Dev list Subject: [mat-dev] Huge heaps I'm looking at bug 372548: ArrayIndexOutOfBoundsException with huge dump https://bugs.eclipse.org/bugs/show_bug.cgi?id=372548 I'm working on a fix to one problem - the 1 to N indexes didn't cope with more than 2^31 outbound references in the whole snapshot. It used an int returned from the header index to index into the body index of all the outbound references. I hope to be able to commit a fix shortly. What are the biggest heaps we need to deal with, in terms of objects or total outbound references? What other restrictions are there for large dumps? Do we need a LongIndex1N which can have more than 2^31 outbounds in total? Do we need more than 2^31 objects? Currently object id < 2^31, i.e. signed int We could defined object id as being unsigned. Possible problems include: Identifier.reverse - a negative number is returned for not found inbounds - where we temporarily encode some refs as negative int SnapshotInfo.getNumberOfObjects() int IClass.getNumberOfObjects() int IClass.getObjectIds() int [] Snapshot.getOutboundReferentIds() SetInt can't hold enough ints int [] Snapshot.getOutboundReferentIds(int[] objectIds, IProgressListener progressListener) - can't return more than 2^31 items int [] Snapshot.getInboundReferentIds(int[] objectIds, IProgressListener progressListener) - can't return more than 2^31 items Do we need to expose a IntIndexReader which can be indexed by unsigned int / longs for > 2^31 entries? Do we need to make the InboundWriter work with huge dumps. It splits the refs into separate log files, but can the contents of the log files get too big to sort as int arrays? Can we save memory on building indices, doing the GC, rebuilding indices, calculating dominator tree? Andrew Johnson Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU _______________________________________________ mat-dev mailing list mat-dev@xxxxxxxxxxx https://dev.eclipse.org/mailman/listinfo/mat-dev ]]> Wed, 14 Mar 2012 16:07:31 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00325.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum) [mat-dev] Huge heaps http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00324.html I'm looking at bug 372548: ArrayIndexOutOfBoundsException with huge dump https://bugs.eclipse.org/bugs/show_bug.cgi?id=372548 I'm working on a fix to one problem - the 1 to N indexes didn't cope with more than 2^31 outbound references in the whole snapshot... I'm looking at bug 372548: ArrayIndexOutOfBoundsException with huge dump https://bugs.eclipse.org/bugs/show_bug.cgi?id=372548 I'm working on a fix to one problem - the 1 to N indexes didn't cope with more than 2^31 outbound references in the whole snapshot. It used an int returned from the header index to index into the body index of all the outbound references. I hope to be able to commit a fix shortly. What are the biggest heaps we need to deal with, in terms of objects or total outbound references? What other restrictions are there for large dumps? Do we need a LongIndex1N which can have more than 2^31 outbounds in total? Do we need more than 2^31 objects? Currently object id < 2^31, i.e. signed int We could defined object id as being unsigned. Possible problems include: Identifier.reverse - a negative number is returned for not found inbounds - where we temporarily encode some refs as negative int SnapshotInfo.getNumberOfObjects() int IClass.getNumberOfObjects() int IClass.getObjectIds() int [] Snapshot.getOutboundReferentIds() SetInt can't hold enough ints int [] Snapshot.getOutboundReferentIds(int[] objectIds, IProgressListener progressListener) - can't return more than 2^31 items int [] Snapshot.getInboundReferentIds(int[] objectIds, IProgressListener progressListener) - can't return more than 2^31 items Do we need to expose a IntIndexReader which can be indexed by unsigned int / longs for > 2^31 entries? Do we need to make the InboundWriter work with huge dumps. It splits the refs into separate log files, but can the contents of the log files get too big to sort as int arrays? Can we save memory on building indices, doing the GC, rebuilding indices, calculating dominator tree? Andrew Johnson Unless stated otherwise above: IBM United Kingdom Limited - Registered in England and Wales with number 741598. Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU ]]> Tue, 13 Mar 2012 10:48:32 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00324.html andrew_johnson@xxxxxxx (Andrew Johnson) Re: [mat-dev] FW: [cross-project-issues-dev] jobs having more than 15 old builds http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00323.html Makes sense. Although I don't think that we really have a guarantee that they will be kept on Hudson forever. With next release I'll mark the build to be kept and will save the junit/Findbugs results elsewhere. > I wasn't worried about the build outputs, but though it would be > useful to see the build logs and FindBugs and JUnit results to > compare to the current builds. I think those are the only things > that get saved for the previous builds. Makes sense. Although I don't think that we really have a guarantee that they will be kept on Hudson forever. With next release I'll mark the build to be kept and will save the junit/Findbugs results elsewhere. ]]> Thu, 08 Mar 2012 15:45:37 GMT http://dev.eclipse.org/mhonarc/lists/mat-dev/msg00323.html krum.tsvetkov@xxxxxxx (Tsvetkov, Krum)