[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
Re: [science-iwg] Few considerations from the yesterday meeting
|
Hi Mario,
Thank you for the very interesting summary and points-of-view.
I must first of all apologize for the somewhat chaotic intro to Triquetrum.
I was tired from traveling and was not well prepared for the kind of
"sales pitch" format ;-)
To address your remarks below, in the context of Triquetrum :
1. As I explained, a large part of the internal architecture is
inherited from a predecessor project called Passerelle.
This also uses Ptolemy II as its "engine", and mainly targets server
clusters as deployment platform for production usage.
Workflows get launched, hundreds of thousands a day, via remote service
invocations, scheduled jobs etc. (and only sometimes by user triggers)
It has been developed as OSGi bundles since 10 years or so allowing us
to package it depending on the target environment needs.
One incarnation is using a plain personal GUI in Swing (derived from
Ptolemy's Vergil). It can do workflows locally or connect to a server to
edit,configure models, run them on the server and get progress info back.
The other extreme (which is not open source though) is a complete
web-based management GUI + web-based editor etc.
These approaches share most of their modules, but have different
interface/view components on top of that (a bit like the Sandia
packaging approach from the meeting yesterday).
The goal for Triquetrum in this respect would be to contain all required
service APIs and core implementations. But I don't think currently that
the web-based interface would be rebuilt in there (unless we can get
significant interest and collaboration to get that done). I would try to
reach a situation where the RCP application could act as management tool
and editor in a kind of client/server setup.
2. About development languages : we're natively Java-based, but
integration with R, Python/Jython, JavaScript, Drools, ... is available
in different ways in Ptolemy, Passerelle and related Science projects.
Python is on the list as the first "non-JVM" integration in Triquetrum.
The basis could be the work we've done in the past to get that working
on Passerelle workflow servers at ESRF.
That code has been prepared for reuse, this will be discussed this week
with Jonah Graham (Kichwa Coders) who has deep knowledge on Java/Python
integration and on the related Science projects.
FYI, the screenshots of the beamline experiment that you refer to, were
completely based on using a workflow definition as a coordination of
Python scripts that they already had.
And their custom GUI to control the beamlines is also Python and Qt if
I'm not mistaken, so we're surrounded by it! ;-)
3. The Triquetrum slides will be my talk at EclipseCon Toulouse
tomorrow. I'll make them available from our Wiki, and I guess Eclipse
will also upload them on the EclipseCon site.
Regards
erwin
Op 07/06/2016 om 09:23 schreef Mario Valle:
Dear group,
thanks for the yesterday meeting! It was really informative and
helpful to me.
One of my jobs here at CSCS is to find, if it exists, a workflow
manager that could be adopted for applications like the Material
Cloud, the NExT VRE European project, or in general for the kind of
computational experiments done at CSCS. The second goal is to
understand what these systems need to provide to become the generic
workflow management system that could be offered as a service by CSCS.
That said, I should confess that I “kindly hate” workflow managers
that do everything locally behind a GUI. I found them limited and not
very user friendly. Maybe it is better to call them, as KNIME does,
“analytics platforms”. For these reasons, I was pleasantly surprised
to hear about the Sandia work and their split desktop / HPC workflow
“console”. Also the Triquetrum from the slides (contrasted to the one
downloaded from the Eclipse project) seems heading in this direction.
The second thing I discovered yesterday is that integration with HPC
machines and workload managers is possible. Seems trivial, but WF
managers like FireWorks have a quite clumsy way to do this. AiiDA is
in a better shape, but it is inextricably linked to the material
simulation part. On a different path, Taverna introduced the
executable components as Web services and Triquetrum will have REST
services. In my opinion, this is the right way to decouple the
“console” (WF editor and management) from the execution part.
Last consideration is about how to define and make available new
executable components. I grew up with massive doses of C, C++, Shell,
TCL/TK and Python so I tend to stay clear of every WF tool that
requires Java programming, but I welcome Python and R scripting. From
the presentations seems I'm not alone in this request.
To make complete my learning I would like to ask one thing. Is it
possible to access the slides presented yesterday? And to have
pointers to the public tools mentioned? Forgive me if I have
misunderstood or forgot something from yesterday, but with Socrates:
“I know that I do not know”.
See you!
mario