Thank you Alex!!
We really appreciate your efforts here!
Would you be willing to create a stem wiki page on kml/google earth
integration documenting what you have been able to do for others?
I'm not sure about your application but another short term fix might
be to run STEM and generate the kml files as step 1 and play back on
google earth as step 2.
or,... maybe get a massive server with lots of cores so you can up the
number of threads.
Anyway please let us know. Thank you for looking at the KML display
classes....
Best Regards,
Jamie
IBM Almaden Research Center, 650 Harry Rd.
San Jose, CA 95120-6099
email: jhkauf@xxxxxxxxxx
phone: (408) 927-2477 (tie 457-2477)
From: alex@xxxxxxxxxxxxxx
To: stem-dev@xxxxxxxxxxx,
Cc: matt.davis@xxxxxxxxxxx, James Kaufman/Almaden/IBM@IBMUS
Date: 10/08/2014 08:18 AM
Subject: re: STEM Feature for single KML file animation -- Dev
Questions
-------------------------
Hello James,
No worries... I'm sure you're very busy. I appreciate the time you
have
taken to explain the parameters.
A. Thanks. I will see if I can extract the population data using the
Java code you've included.
B. Also great info here. I'll hunt down the Sequencer object to get
the start and end dates.
C. I wrote: "Why is my simulation's 'cycle' counter jumping
around..."
You were absolutely correct... it turned out to be a threading
problem.
I resolved it by doing the following:
-Increased the STEM simulation step time from 250ms to 750ms
-Set the STEM Process Priority to High in the OS
-Allocated STEM 2GB/2GB of memory on launch
-Slowed the new KML writer so it writes only once per 5 cycles
(rather
than each cycle).
-Reduced the complexity of the simulation... the max output of the
KML
file can only be about 400MB.
I learned that if the single KML file is greater than about 400 MB,
Google Earth will crash when loading it.
So unfortunately, this limits the 'granularity' of KML file
animations.
There is probably a way around this... at present, I write the
Polygons
to represent a county repeatedly (at each simulation step). This
makes
county-level data cause the KML animation file to be very large
(hundreds of megs) .. There is lots of redundant data since the same
polygon coordinates gets written over and over, repeated on each
TimeSpan.
I see three workarounds to the large KML output file sizes... The key
will be to eliminate the KML polygon data redundancy over
TimeSpans...
C.1. Utilize image overlays (jpegs) for countries, regions and
counties
etc... rather than having Google Earth draw STEM's region polygons
over
and over, use colored JPG overlays, shape files, something like that.
C.2. OR: Have GE import STEM county-level coordinates as a resource
'as-needed', and use Google Earth's 'Feature' loading capability to
cache only the data in view. This way it doesn't all get loaded into
memory at once, and might eliminate redundancy.
C.3. OR: Quick and Dirty solution: Reduce the number of points
(resolution) that is used to represent county data when writing to
the
KML file by Ramer–Douglas–Peucker algorithm. So the county
polygons
will not be so complex in the KML, and will take up less space. ie.
Reduce a county polygon from ~50 coordinate pairs to 10 coordinate
pairs.
http://en.wikipedia.org/wiki/Ramer%E2%80%93Douglas%E2%80%93Peucker_algorithm
D. Thanks. I think I will try to use your standard methods (the
Aspect
object, scaling, etc) for coloring the KML output polygons. That way
everything is consistent across the STEM platform. I want the KML
animation to support log scaling , just like the GUI does.
E. Okay. I'm pretty sure the STEM GE Servlet stuff is broken on my
Windows 7 x64 environment. For some reason, the web server has not
ever
run locally (nothing is bound on port 8080 or whatever it is). I
think
this functionality might be broken or deprecated in the current
versions
of Java and Google Earth.
The existing GE functionality in the latest STEM release was able to
do
two major tasks (1) write a directory with one kml file per iteration
--
numbered, or (2) launch Google Earth to load a control.kml file once
per
iteration. I don't think I was able to get the GE Servlet stuff
working at all, but I will test again.
F. > What is the difference between KmlDisplay, KmlDisplayCustom, and
KmlDisplaySelection? Is any of this functionality deprecated? "Some
of
these are really old and may go back to older versions of google
earth."
Okay, I may try to re-factor some of this code. I think
KmlDisplaySelection was meant to display a sub-set of a geographic
area
to KML, but I don't know if it's working. I think some of this code
can
be pulled out, fixed, or refactored, especially if the Servlet stuff
is
actually broken for everyone.
Anyway, I will keep digging and clean up the code as I go. I'd like
to
pull out or fix any deprecated features that are broken.
Also, I'm tempted to convert the KML write functionality to a Logger
object, unless there is a reason not to do that...
Thanks,
Alex (aka Vincent R)
On 2014-10-07 16:38, James Kaufman wrote:
> Vincent,
> Sorry for the slow response.
> Re your questions
>
> > A. Where is the 'county' population data stored? For example, how
> do I go from a key like 'US-NY-36039' (county identifier) to
retrieve
> the original population in the county and the current population in
> the county at a time-step? (to determine number of fatalities , or
> total number of Infections as an absolute value rather than
percent).
>
> if you have the node you can use something like the following
> EList<NodeLabel> labelList = node.getLabels();
> FOR(NodeLabel nl:labelList) {
> IF(nl INSTANCEOF PopulationLabel) {
> PopulationLabel pl = (PopulationLabel) nl;
> DOUBLE population = pl.getCurrentPopulationValue().getCount();
> }
> }
> You should be able to get the node from the graph. (ie
> graph.getNode(nodeURI))
>
> > B. Where can I find a simulation's actual 'start' and 'end' dates
> in String or DateTime format? What object and where is it?
> Get the Sequencer from the scenerio (from the simulation). The
> sequencer has the start and end dates.
> the sequencer has method getStartTime and getEndTime
> they return "STEMtime" objects but STEMtime.getDate() gives you a
> Date.
>
> > C. Why is my simulation's 'cycle' counter jumping around at the
> beginning? It starts at cycle=0, then goes to cycle=2, then to
> cycle=7,8,9,10 etc?
> The gui runs in it's own thread so you are running either a big or
> small simulation. On the menu bar you can go to
window>preferences.
> then >STEM>Simulation Managment and either increase the simulation
> delay so the gui keeps up. Or decrease it if you need.
>
> > D. Is there a 'proper' way to go about scaling an 'I' value to a
> polygon color? Should I just use the Aspect object? Any tips on
this?
>
> I think it just depends what you are trying to show. We usually
> normalize I by the population but you might be interested in small
> values at the start of an epidemic so you might need a scale factor
or
> log scaling.
>
> > E. Are there any known bugs with the GE Servlet stuff? I'm
getting
> some Servlet exceptions (can't connect to local web server). Is
this
> feature broken (the network pipe)? It's not
> important -- just curious.
>
> We have not run this in a very long time. Matt, any ideas on this?
>
> > F. Why are there multiple classes which seem to be involved in
> writing KML files... What is the difference between KmlDisplay,
> KmlDisplayCustom, and KmlDisplaySelection? Is any of this
> functionality deprecated? Some of these are really old and may go
back
> to older versions of google earth.
>
> I'm sorry not to be more helpful....
>
> Best Regards,
> Jamie
>
> IBM Almaden Research Center, 650 Harry Rd.
> San Jose, CA 95120-6099
> email: jhkauf@xxxxxxxxxx
> phone: (408) 927-2477 (tie 457-2477)
