RE: [higgins-dev] Sub-contexts

[Mike McIntosh <mmcintosh@xxxxxxxxx>]

I know this seems obvious to those of you that are elbow deep in this stuff but…

Why do we want to prevent sub-contexts of sub-contexts?

 

Regards,

Michael McIntosh

VP Development

Azigo

 

PS: I really wish outlook made it easier to track who made what comment when comments are embedded in responses…

 

From: higgins-dev-bounces@xxxxxxxxxxx [mailto:higgins-dev-bounces@xxxxxxxxxxx] On Behalf Of Paul Trevithick
Sent: Tuesday, May 25, 2010 7:03 PM
To: Sergey Lyakhov
Cc: higgins-dev developer discussions
Subject: Re: [higgins-dev] Sub-contexts

 

 

On May 24, 2010, at 12:06 PM, Sergey Lyakhov wrote:

Paul,

We have to add to IdAS API the ability to add/find/delete a sub-Context within a Context. I hope SergeyL can suggest

a straw man change.

As I undersntand, we should use the following mapping for NG4J interfaces:

a) Context to NamedGraphSet interface.
b) SubContext to NamedGraph interface.

NamedGraphSet is just a collection of NamedGraphs, so, it is not possible for a SubContest to contain its own SubContexts (only a "root" Context can contain SubContexts).

As a result, I would propose:
1. rename the current IContext interface to something like IBaseContext interface.
2. add ISubContext interface extends  IBaseContext interface.
3. add IContext interface, that extends IBaseContext, with the following methods:
a) Iterator getSubContexts();
b) ISubContext getSubContext(URI subContextID) (where subContextID is a name of an appropriate graph);
c) ISubContext addSubContext(URI subContextID);
d) ISubContext removeSubContext(URI subContextID);
e) ISubContext contextualize(entityid, attributes[]);
f) void contextualize(entityid, attributes[], ISubContext) - moves attributes into passed subcontext;

BTW, because NamedGraph can contain a separate triplets, we can add the following methods to contextualize an individual value:
g) ISubContext contextualize(IEntity, IValue);
h) void contextualize(IEntity, IValue, ISubContext).

 

Looks good.

Valery (and I think Vitaliy) had wanted a way to associate metadata with h:correlation links from A to B (across

contexts c1 and c2). With sub-contexts this is now easy. You just move (contextualize()) this one

entity-attribute-value triple from the default (main) context to the sub-context. Now you can attach whatever

attributes to the sub-context you like.

I do not understand how to use named graphs to attach metadata. Of course, we can "mark" any separate triple with its graph name. But how are you going to attach metadata to that graph? Can you provide me with an example of such metadata?

 

Here is an example. Imagine there are two h:correlation links:

 

:Foo

            h:correlation :Bar, :Baz;

 

If we put these two links into two separate contexts (C1 and C2) we have:

 

:C1 

{

            :Foo

                        h:correlation :Bar;

}

:C2 

{

            :Foo

                        h:correlation: :Baz;

}

 

And now we can attach some metadata about C2. Perhaps we want to say that the h:correlation link it contains is Valery's "favorite" link. This can be done in C1 or C2 (or any other sub or main context):

 

In C1:

            :C1 

            {

                        :Foo

                                    h:correlation :Bar;

                        :C1

                                    :valerys-favorite "True"^^xsd:boolean

            }

            :C2 

            { 

                        :Foo

                                    h:correlation: :Baz;

            }

 

In C12

            :C1 

            {

                        :Foo

                                    h:correlation :Bar;

 

            }

            :C2 

            { 

                        :Foo

                                    h:correlation: :Baz;

                        :C1

                                    :valerys-favorite "True"^^xsd:boolean

            }

 

From the other hand, we could use SubContexts for Access Control.

 

Yes, sub-contexts are ways to keep things tidy--which to me means keeping "metadata" (like access control, change events, provenance, etc) in their own sub-contexts and away from the main data graph.

The good news is that our main CP on the PDS (aka Attribute Service) backend is backed by Jena.

And there exists a version of NG4J [4] that already supports the necessary semantics.

Actually, we need to write the context from a scratch, because NG4J uses absolutely different approach (NamedGraphs/Quards/Triples/Nodes), despite it uses/supports some Jena classes/interfaces.

Also I see the following problems with using NG4J:
1. NG4J does not support Jena OntModel interface (at least I not found how to work with OWL), only RDF model is supported.
2. We always need to work with Quads, not with a model.
3. NG4J uses its own serialization (TriX). In case of JDBC, it inferior even to Jena persistence way. E.g., it does not support data length > 2000 symbols. Also, I found a bug in de.fuberlin.wiwiss.ng4j.db.specific.DbCompatibility. It does not close DB Statements in case of any SQL error (like long data), as a result it can not  process any query after the connection exhausted the limit of non-closed statements.
4. NG4J does not support transactions (at least I not found how to do that).

 

This is unfortunate. If it's a lot of work to implement NG4Jena and then when we're done we only have a toy implementation that won't even support a limited scale, then NG4Jena seems like a waste of time. 

 

Options:

            1. Get on the NG4Jena mailing list and ask for advice [We should do this no matter what]

            2. See if there are alternative open source quad stores with better performance/scalability

            3. Explore non-relational open source graph data bases (e.g. Infogrid, Neo4j, etc.)

            4. Use an XDI native store

            5. Decide to defer implementation of sub-contexts

 

On (5) above: without sub-contexts:

            * we will have to have explicit links from "regular" objects to "metadata" (e.g. provenance)

            * data & metadata will be mixed together

 

 

Thanks,
Sergey Lyakhov

On Tue, 6 Apr 2010 18:58:24 -0400
Paul Trevithick <ptrevithick@xxxxxxxxx> wrote:

Folks,

 

I strongly believe that we should add sub-contexts (within a parent context) to the CDM. I have updated the

definition of Context [1]. I copy the key bit here (from "CDM 2.0" onwards): Is a set of one or more Entities

Is a special kind of entity identified by a ContextId

Have zero or more Attributes.

Has a schema (ontology) that describes kinds of Entities and Attributes that an instance of this Context contains.

Has its own security and access control policy

CDM 2.0: May (strictly) contain zero or more sub-Contexts. A sub-Context is a Context with these restrictions:

It is entirely contained within its parent Context

Its set of entities is any subset of its parent's set of entities.

Inherits its schema from its "parent" context

Inherits its security and access control policy from its "parent" context

I have also updated the bullet on Contexts here [2]. And I have updated these paragraphs [3].

 

IMPLICATIONS

 

(1) Enhancement to IdAS.

 

We have to add to IdAS API the ability to add/find/delete a sub-Context within a Context. I hope SergeyL can suggest

a straw man change. The good news is that our main CP on the PDS (aka Attribute Service) backend is backed by Jena.

And there exists a version of NG4J [4] that already supports the necessary semantics.

 

I think (for convenience) we should add a new API called "subcontext = contextualize(entityid, attributes[])" that

would take an entity and zero or more of its attribute type URIs (and we need to specify down to the individual value

level too) and move them into a subcontext and return the id of the subcontext. Maybe we need a simpler "move" method

too in case the subcontext we want to move stuff into already exists.

 

What I like about this change to IdAS is that it is really clean. We don't have to introduce "Statement" classes

(triples, etc.) into the IdAS API. The contextualize method is all we need. After that we're back to Contexts,

Entities, Attributes and Values all over again.

 

(2) Statements about Statements

 

With sub-contexts (and esp with the contextualize() method) we now have a way of taking statements being made in one

context (or sub-context) and moving them into a sub-context. Since this sub-context itself has an entityid we can

attached attributes to it ("inside" the subcontext so to speak) or we can make statements where the value (RDF

object) is the subcontext ("outside" the subcontext in the main "dataset" context).

 

Valery (and I think Vitaliy) had wanted a way to associate metadata with h:correlation links from A to B (across

contexts c1 and c2). With sub-contexts this is now easy. You just move (contextualize()) this one

entity-attribute-value triple from the default (main) context to the sub-context. Now you can attach whatever

attributes to the sub-context you like.

 

I had proposed that we handle password history in a password specific way. But after talking with Mike, I'm

re-thinking (yet again) if we could implement history metadata in a 100% generic ( non-pwmgr-specific) way.

 

(3) More similar to XDI

 

With this change we make supporting XDI easier. This will be important because there is a move afoot to sort of

standardize data sharing using XDI.

 

--Paul

 

 

[1] http://wiki.eclipse.org/Context

[2] http://wiki.eclipse.org/Context_Data_Model_2.0#Key_Concepts

[3] http://wiki.eclipse.org/Context_Data_Model_2.0#Relationship_to_RDF

[4] http://www4.wiwiss.fu-berlin.de/bizer/ng4j/