Re: [rdf4j-dev] New Memory Store - Planning

[Håvard Ottestad <hmottestad@xxxxxxxxx>]

So the main goal is to make a faster and more configurable in-memory database for rdf4j. 

Improving or rebuilding the TreeModel to be configurable and support a B+ tree seems like a great project, but for now it’s too much work to make those improvements useful with regards to support for reasoning and shacl.

I’ll keep my focus on building a new MemoryStore instead, it’ll take quite some time to iron out all the edge cases in the B+ tree. Hopefully I can handle transactions by limiting the level to NONE and using locking in the short term and then either go with copy-on-write or with a stratified tree later on.

Håvard

> On 19 Jul 2017, at 16:53, James Leigh <james.leigh@xxxxxxxxxxxx> wrote:
> 
> The SAIL API came first. The Model was introduced as a way of passing
> triples around and is not intended for use as a datasource. If
> reasoning is important likely you are looking for a datasource and
> should use the SAIL API. If that's the case I suggest you plan on
> implementing transactions as a SAIL implementation without transactions
> may not be that useful to the rest of the community. If you go with a
> pessimistic locking mechanism you should include a fail-fast feature as
> dead locks are difficult to debug.
> 
> Regards,
> James
> 
> On Wed, 2017-07-19 at 12:51 +0200, Håvard Ottestad wrote:
>> A point for using the sail api is that it would then support rdfs
>> reasoning and shacl (in the future) since both of these are sails
>> too. 
>> 
>> Håvard
>> 
>> 
>>> 
>>> On 19 Jul 2017, at 09:53, Håvard Ottestad <hmottestad@xxxxxxxxx>
>>> wrote:
>>> 
>>> Could wrap it in locks and set the highest transaction level to
>>> “none”. Make .rollback() throw an unsupported operation.
>>> 
>>> It would just be much nicer if there, in general, was one api for
>>> accessing graph data in rdf4j instead of the two that there are
>>> today. Layering the api, depending on what the underlying datastore
>>> supported would be much nicer. Eg. if the underlying datastore
>>> didn’t support transactions, then it’s connection wouldn’t have the
>>> begin() or commit() methods (using generics).
>>> 
>>> Which came first, model or sail?
>>> 
>>> Håvard
>>> 
>>> 
>>> 
>>>> 
>>>> On 18 Jul 2017, at 21:16, James Leigh <james.leigh@xxxxxxxxxxxx>
>>>> wrote:
>>>> 
>>>> The Sail interface includes transactions, so it would not be
>>>> appropriate. However, you can utilize all the query evaluation
>>>> libraries that are used by Sail impls without needing a
>>>> SailConnection.
>>>> Take a look at EvaluationStrategy. By implementing a TripleSource
>>>> and EvaluationStatistics you should be good to write a helper to
>>>> perform common optimizations. Look at the SailSourceConnection as
>>>> an
>>>> example.
>>>> 
>>>> Regards,
>>>> James
>>>> 
>>>> On Tue, 2017-07-18 at 09:29 +0200, Håvard Ottestad wrote:
>>>>> 
>>>>> What would you think about a sail wrapper for models? 
>>>>> 
>>>>> Wouldn't be thread safe or support transactions. But then you
>>>>> would
>>>>> get SPARQL support for free.
>>>>> 
>>>>> Håvard
>>>>> 
>>>>> 
>>>>> On Mon, Jul 17, 2017 at 8:59 PM, James Leigh <james.leigh@ontot
>>>>> ext.co
>>>>> m> wrote:
>>>>>> 
>>>>>> On Mon, 2017-07-17 at 16:13 +0200, Håvard Ottestad wrote:
>>>>>>> 
>>>>>>> Would be so much more pleasant to make a memory store that
>>>>>>> didn’t
>>>>>>> support transactions or parallel updates.
>>>>>> 
>>>>>> What about creating a better TreeModel? And/Or adding SPARQL
>>>>>> support to
>>>>>> Models?
>>>> _______________________________________________
>>>> rdf4j-dev mailing list
>>>> rdf4j-dev@xxxxxxxxxxx
>>>> To change your delivery options, retrieve your password, or
>>>> unsubscribe from this list, visit
>>>> https://dev.eclipse.org/mailman/listinfo/rdf4j-dev
>> _______________________________________________
>> rdf4j-dev mailing list
>> rdf4j-dev@xxxxxxxxxxx
>> To change your delivery options, retrieve your password, or
>> unsubscribe from this list, visit
>> https://dev.eclipse.org/mailman/listinfo/rdf4j-dev
> _______________________________________________
> rdf4j-dev mailing list
> rdf4j-dev@xxxxxxxxxxx
> To change your delivery options, retrieve your password, or unsubscribe from this list, visit
> https://dev.eclipse.org/mailman/listinfo/rdf4j-dev