[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
[smila-user] Metadata in Pipelets not associated with Records


I'm developing several pipelets to interact with a webservice. I'm wrapping most methods into separate pipelets in order to give the SMILA user different tools. I'm however unsure what would be the "best practise" when it comes to actions that are not necessarily associated with any records at all.

Our webservice allows to upload data read from records. This is working fine. Besides uploading the method allows triggering the start of processing the previously uploaded data. Processing (which is happening completely outside of SMILA) takes several hours. So, one workflow could be: "Take this amount of records, and upload the appropriate data". Another workflow could be: "Take this amount of records, upload the appropriate data and trigger processing", or "Just trigger processing with the already uploaded data". The initial idea is to put the uploading part and the triggering part into different pipelets. The triggering pipelet however is completely independent on the record ids that were passed into it (as it would simply call a webservice method and be done). I guess this could easily work by simply ignoring the records.

Another method of the webservice lists all uploaded data and another method can be used to remove specific data items. I'd like to have the listing and the deletion methods be in separate pipelets. For a workflow like "Delete all old data, upload new data, trigger processing", I would need to create a list of "old data" and pass it to the deletion pipelet. Both of them could be more or less ignorant of the records passed to them, but I would need to pass on the list from the first to the second pipelet. Is there a distinguished way to pass on these general kind of metadata which are not really associated with a record? Or is SMILA on the pipelet level just not designed for this?

Another question that popped to my mind is whether it is possible to just trigger a workflow with need of the crawler to find new files? So say if I want to trigger the processing every night at 2:00, could I write a little cronscript which would somehow access SMILA (through the REST API?) and kicks off the worklow with no records at all? How do I trigger this? A link to appropriate documentation would be sufficient.

Finally, I have code which can monitor the status of the processing job given an id which is returned when triggering this processing (e.g. from the above mentioned pipelet). Again, the pipelet triggering the processing would need to store this id somewhere from which the monitoring code could access it. Is it possible at all to build something like: "Monitor status and if status is 'finished', trigger workflow X with record set Y"? Or is SMILA simply not designed for this?

Stephan Mehlhase

European Media Laboratory GmbH
Schloss-Wolfsbrunnenweg 35
69118 Heidelberg

Amtsgericht Mannheim / HRB 335719
Managing Partner: Dr. h. c. Dr.-Ing. E. h. Klaus Tschira, Scientific and Managing
Director: Prof. Dr.-Ing. Andreas Reuter


Testen Sie auch unsere neue Android Applikation zur Mediensteuerung: