Re: [stem-dev] Effects of enabling CDO for STEM

[Matthew Davis <mattadav@xxxxxxxxxx>]

Hi Tero,

Sorry for the delayed reply.

Took a look at your scenarios.  Those are all things we've discussed and 
would be nice features.  The cost of constantly sending the canonical 
graph over the wire is probably too high, especially when the vast 
majority of the data isn't all that useful to the client while the 
simulation is running.  Would there be a way to restrict or defer (or only 
request small pieces) of the data from the CDO objects, with the rest 
remaining on the server?  For a visualization example, you would only need 
values for the labels to be sent back and forth.  A text data file 
containing a label value for all geographies weighs in at ~1 MB... with 
binary encoding, it'd be significantly smaller.  If we fix the update 
intervals in the client, and with limited data exchange, this doesn't seem 
too unreasonable.

The second part is what I find more interesting.  CDO in audit mode 
allowing for "quick" random access to the canonical graph history is 
something that is badly needed.  There are data management issues, and I'm 
not sure how robust CDO audit is, especially with the volume of data being 
generated.  However, it could help with the problem of over-the-wire data 
back to the client if the client became asynchronous to the simulation. In 
my opinion, this is the more valuable thing for STEM right now... and a 
feasibility test to see if CDO can handle the data load would help a 
bunch.

Thanks for the great work so far!

-Matt

stem-dev-bounces@xxxxxxxxxxx wrote on 11/02/2010 12:08:36 PM:

> From: Tero Parviainen <tero.parviainen@xxxxxxxxxx>
> To: STEM developer mailing list <stem-dev@xxxxxxxxxxx>
> Date: 11/02/2010 12:09 PM
> Subject: Re: [stem-dev] Effects of enabling CDO for STEM
> Sent by: stem-dev-bounces@xxxxxxxxxxx
> 
> Hi,
> 
> I'm happy to report some progress on the CDO front. I've just 
> successfully run a "remote simulation", where the simulation graph 
> was transferred over the network with CDO to a remote machine, 
> calculated there, and the results sent back live back to the client.
> 
> I wrote a wiki page about the approach and some possible implications: 
> http://github.com/teropa/stem/wiki/STEM-Server-Scenarios
> 
> Br,
> Tero
> 
> On Oct 6, 2010, at 7:42 PM, Tero Parviainen wrote:
> 
> Hi Stefan,
> 
> There indeed is a noticeable performance penalty, but I'm still 
> optimistic something can be done about that.
> I'm sorry to say I'm a bit tied up at the moment and aren't able to 
> join the call, but I'll elaborate on the mailing list later, and 
> hope to speak to you next week.
> 
> Br,
> Tero
> 
> ________________________________________
> From: Stefan Edlund [edlund@xxxxxxxxxxxxxxx]
> Sent: 05 October 2010 23:48
> To: Tero Parviainen
> Cc: stem-dev@xxxxxxxxxxx
> Subject: Re: [stem-dev] Effects of enabling CDO for STEM
> 
> Hi Tero,
> 
> this is a great test, thank you very much for trying this out! If I 
> understood your tests correctly, there seems to be a pretty severe 
> performance penalty enabling CDO even just running standalone?
> 
> I added an item for tomorrow on the STEM agenda if you can make it. 
> Perhaps you can reiterate and clarify what you discovered so far.
> 
> Regards,
> / Stefan
> 
> 
> Stefan Edlund
> Public Health and Computer Science Research
> IBM Almaden Research Center
> (408) 927-1766 edlund@xxxxxxxxxxxxxxx
> "Bad news is good news. Good news is no news. No news is bad news."
> 
> [cid:1__=07BBFD20DFE277A68f9e8a93df938@xxxxxxxxxx]Tero Parviainen 
> ---10/03/2010 08:29:32 AM---> I'm not seeing any noticeable 
> performance degradation after the CDO refactoring, but I haven't > a
> 
> 
> From:
> Tero Parviainen <tero.parviainen@xxxxxxxxxx>
> 
> To:
> STEM developer mailing list <stem-dev@xxxxxxxxxxx>
> 
> Date:
> 10/03/2010 08:29 AM
> 
> Subject:
> Re: [stem-dev] Effects of enabling CDO for STEM
> 
> Sent by:
> stem-dev-bounces@xxxxxxxxxxx
> 
> ________________________________
> 
> 

> I'm not seeing any noticeable performance degradation after the CDO 
> refactoring, but I haven't
> actually measured this.
> 
> I got around to doing some performance tests on both CPU and memory 
> usage, and wrote about the results here: http://github.com/teropa/
> stem/wiki/Effects-of-CDO-on-STEM-Performance

> Here are some of my main observations
> 
> I noticed one additional caveat in converting the code to CDO: All 
> default values need to be set in the "Default value literal" 
> property of the EMF model and generated from that, because otherwise
> EMF does not know about them and they won't get used. Of course, 
> they can alternatively be set manually during object construction. 
> There were a few cases where default values were annotated 
> "@generated NOT" (e.g. 
> SequentialSequencerImpl.TIME_INCREMENT_EDEFAULT), and I changed 
> those in my cdo branch.
> 
> Br,
> Tero
> 
> ________________________________________
> From: stem-dev-bounces@xxxxxxxxxxx [stem-dev-bounces@xxxxxxxxxxx] On
> Behalf Of Tero Parviainen [tero.parviainen@xxxxxxxxxx]
> Sent: 29 September 2010 17:48
> To: STEM developer mailing list
> Subject: [stem-dev] Effects of enabling CDO for STEM
> 
> Hi,
> 
> I've been exploring the practical consequences of CDO-enabling the 
> STEM EMF model. I did the genmodel conversion by following the 
> instructions in the CDO wiki (http://wiki.eclipse.org/
> Preparing_EMF_Models_for_CDO), and the code is in my Github repo at 
> http://github.com/teropa/stem/tree/cdo.
> 
> Here are some of my main observations:
> 
> - CDO requires we use either dynamic or reflective feature 
> delegation in EMF models. The main
> effect this has to the current application code is that direct field
> access cannot be used in EMF
> models. Thus, all places where fields are accessed directly in non-
> generated code have to be
> changed to getter/setter access (this.canonicalGraph -> 
> getCanonicalGraph()). This requires
> quite a large amount of changes in the code, but they are for the 
> most part simple to make.
> 
> - EMF seems to prefer identity semantics for implementing equals/
> hashCode, and CDO actually
> enforces this by making them final in CDOObjectImpl. There are a few
> cases in the STEM code
> where equals/hashCode have been overriden to use value semantics 
> (most notably STEMTime),
> and this doesn't work with CDO. Places where these objects are 
> tested for equality or used in
> HashMaps/HashSets would have to be changed.
> 
> - I'm not seeing any noticeable performance degradation after the 
> CDO refactoring, but I haven't
> actually measured this. I also haven't yet used any of the 
> distribution capabilities of CDO, so
> I cannot yet comment on how well that works.
> 
> Br,
> Tero
> 
> _______________________________________________
> stem-dev mailing list
> stem-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/stem-dev
> _______________________________________________
> stem-dev mailing list
> stem-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/stem-dev
> 
> 
> _______________________________________________
> stem-dev mailing list
> stem-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/stem-dev
> _______________________________________________
> stem-dev mailing list
> stem-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/stem-dev