[rdf4j-dev] New Memory Store - Planning

["Håvard M. Ottestad" <hmottestad@xxxxxxxxx>]

Hi,

I’m look at creating a new Memory Store that is configurable so that it can be optimised for any scenario.

Here is what I’m thinking of doing:

 - B+ tree indexes (because they support range queries)
 - Thread based indexing (async)
 - Deletes in diff index (merged when index.size > some max)
 - Hashmap index for exists(triple1) queries

I don’t want to have transactional support. I use transactions with disk based databases, but the memory store I usually only use for embedded purposes that are single threaded. 

Scenarios I want to support:
 - read heavy
 - write heavy
 - transforms (deserialise, query, update, serialise)

In general, are there any recommendations or requirements that others might have before I get too committed?

I also know that a lot of triple stores convert IRIs and literals to a hash/integer so that the hash/integer is stored in the indexes and the IRIs/literals are stored in a lookup table. I’m considering doing this too, but it might not be of much benefit unless I migrate to manual memory management using the unsafe library. Any thoughts on using sun.misc.Unsafe?

Håvard