[cosmos-dev] COSMOS DC Assemblies and TPTP DMS

[Mark D Weitzel <weitzelm@xxxxxxxxxx>]

If you have not had an opportunity to
review the Data Management Service document on the TPTP wiki site (http://wiki.eclipse.org/index.php/TPTP_DMS),
please find a few moments to read it through.  I was recently looking
this over in an attempt to reconcile the document that Don started regarding
our data collection service with what is there in TPTP.  We are each
attempting to accomplish a similar task, and by comparing the two documents
we will be able to identify how the technical pieces relate to one another,
the contracts between the different layers and components, and the concrete
relationship between the two projects.   Looking at what is there
in each document so far, I would make the loose comparison of the following
components:

COSMOS        TPTP
Data Source        Event
Source

Data Filter
& Transform      
 Event Parser

Data Sink        Store
Specific Loader

Each "stack" follows the same
processing pattern and is comprised of the same set of components:
1. An event comes in and some logic
is performed to determine the "kind" or "type" of event.
 This determines the set of POJOs that get created to hold the event
data.  
2. These POJOs are registered based
on the "kind" or "type" of event.  For example,
if the event is a CBE, we get one set of objects, if it's statistical data,
we get a different set of objects.  In COSMOS, this is the Data Filter
& Transform and TPTP, the Event Parser
3. Once we have a set of POJOs, we find
the data store(s) (again based on registration) that will persist them.
 COSMOS, Data Sink, TPTP, Store Specific Loader.

Between each of these processing steps
is an API contract.  Between the Data Filter/Event Parser and the
Data Sink/Store Specific Loader, there is a set of POJOs that serve as
this contract.  For example, in the case of statistical models, this
is where we would consider using the SDMX standard that Chris pointed us
to the other day.  For WEF, this will be the apache muse WEF implementation.
 We still need to work out the other set of POJOs that we need--we
have not really started looking at what we've been calling the "resource",
which gave us the topology graph in the demo.   On the call with the
TPTP team last week, we agreed to share the same POJO models.  At
each contractual API boundary, there is the idea of registration so the
components can be bound together dynamically.  COSMOS calls this a
"Data Collection Assembly" and in TPTP, this is the DMS itself.

We've been working since January to
make sure we position the technology offered in the two projects effectively.
  As the TPTP DMS is starting be flushed out, we need to re-visit
this to understand how the two projects will work together.  For example,
when the DMS is fully implemented, how much of it will COSMOS reuse?  COSMOS
has requirements on loose coupling and dynamic binding and assembly.  Will
the TPTP DMS support this model?  This would be a good use case to
determine how other management infrastructure will fit into the COSMOS
framework.   

The next step is to ask the TPTP team
to present the DMS.  Regardless of project boundaries etc. I'd like
to make sure we have a complete shared understanding of each others requirements.
 For example, since COSMOS is not shipping a database, we are going
to use TPTP "out of the box".  However, given the DMS work,
we need to ensure the requirements we have will be satisfied by TPTP.  Joel's
work at plugging in some TPTP components is a good start and needs to be
extended to include the DMS.

Please let me know your thoughts when
you get a moment.  I'll ping the TPTP team and see when we can get
something setup.

Thanks!


-Mark W.




Mark Weitzel | STSM | IBM Software Group | Tivoli | Autonomic Computing
| (919) 543 0625 | weitzelm@xxxxxxxxxx