[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[smila-user] Metadata in Pipelets not associated with Records
- From: Stephan Mehlhase <stephan.mehlhase@xxxxxxx>
- Date: Thu, 02 Feb 2012 08:16:28 +0100
- Delivered-to: email@example.com
- Organization: EML GmbH
- User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:9.0) Gecko/20111229 Thunderbird/9.0
I'm developing several pipelets to interact with a webservice. I'm
wrapping most methods into separate pipelets in order to give the SMILA
user different tools. I'm however unsure what would be the "best
practise" when it comes to actions that are not necessarily associated
with any records at all.
Our webservice allows to upload data read from records. This is working
fine. Besides uploading the method allows triggering the start of
processing the previously uploaded data. Processing (which is happening
completely outside of SMILA) takes several hours. So, one workflow could
be: "Take this amount of records, and upload the appropriate data".
Another workflow could be: "Take this amount of records, upload the
appropriate data and trigger processing", or "Just trigger processing
with the already uploaded data". The initial idea is to put the
uploading part and the triggering part into different pipelets. The
triggering pipelet however is completely independent on the record ids
that were passed into it (as it would simply call a webservice method
and be done). I guess this could easily work by simply ignoring the records.
Another method of the webservice lists all uploaded data and another
method can be used to remove specific data items. I'd like to have the
listing and the deletion methods be in separate pipelets. For a workflow
like "Delete all old data, upload new data, trigger processing", I would
need to create a list of "old data" and pass it to the deletion pipelet.
Both of them could be more or less ignorant of the records passed to
them, but I would need to pass on the list from the first to the second
pipelet. Is there a distinguished way to pass on these general kind of
metadata which are not really associated with a record? Or is SMILA on
the pipelet level just not designed for this?
Another question that popped to my mind is whether it is possible to
just trigger a workflow with need of the crawler to find new files? So
say if I want to trigger the processing every night at 2:00, could I
write a little cronscript which would somehow access SMILA (through the
REST API?) and kicks off the worklow with no records at all? How do I
trigger this? A link to appropriate documentation would be sufficient.
Finally, I have code which can monitor the status of the processing job
given an id which is returned when triggering this processing (e.g. from
the above mentioned pipelet). Again, the pipelet triggering the
processing would need to store this id somewhere from which the
monitoring code could access it. Is it possible at all to build
something like: "Monitor status and if status is 'finished', trigger
workflow X with record set Y"? Or is SMILA simply not designed for this?
European Media Laboratory GmbH
Amtsgericht Mannheim / HRB 335719
Managing Partner: Dr. h. c. Dr.-Ing. E. h. Klaus Tschira, Scientific and
Director: Prof. Dr.-Ing. Andreas Reuter
Testen Sie auch unsere neue Android Applikation zur Mediensteuerung: