[mat-dev] Off-heap based parsing for

[Jason Koch <jkoch@xxxxxxxxxxx>]

Andrew, Krum, & all,

I have taken a first pass, and created a ticket / proposal / request
for comment on a patch that allows for off-heap storage to be used
during the parsing stages:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=572512

As example, it allows me to use Xmx/Xms5g to parse a 20GB file, where
prior to that it fails.

There are a lot of comments / notes on the ticket, so I won't drain it
fully here. It has the tradeoff that the user has to have sufficient
physical RAM and/or can spill to a storage device. IMO this is
beneficial as it allows parsing of heaps previously not possible, but,
would welcome feedback and use case insights. Also, worth noting, we
may be able to extend this approach to more areas in parsing and save
even more memory.

Thanks
Jason