[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
Re: [smila-user] Push records from Filesystem Crawler to a running Job
|
Hi,
> I'm trying to crawl a directory on file system (using the provided File Crawler Job), containing 29 video files,
> in order to process them in a further pipeline.
> The pipeline called FrameSegmentationPipeline is contained within a workflow called FrameSegmentationWorkflow,
> nd the related Job is > called FrameSegmentationJob.
> Here the jsons:
[... snip ...]
> It seems that the the files were crawled but then are not pushed correctly to the FrameSegmentation job, thus they never reach the
> pipeline. I checked with JConsole and I found that the Pipeline is never invocated.
> Am I missing any configuration? Or am I simply doing something wrong?
The configuration looks OK. It looks like that the DeltaChecker did filter out all records because they have not changed since a previous run, so the fileFetcher and updatePusher workers did not get any input and they are no entries for them in the job run data logged after the job has finished. You can clear it's storage by sending a "DELETE /ias/importing/delta/file" before you re-start the crawl job. Alternatively, if you are using a current version of SMILA (less than two weeks old) you can disable the Delta checking by adding a parameter "deltaImportStrategy": "disabled" to the crawl job. See http://wiki.eclipse.org/SMILA/Documentation/Importing/Concept#Delta_Delete for details.
Cheers,
Juergen.
