AW: [smila-dev] Inconsistent Connectivity state

[<Daniel.Stucky@xxxxxxxxxxx>]

Hi Andreas,

As soon as a record has entered the Queue it has savely entered SMILA and we can ensure that it is processed. This is more important than ensuring minimal processing time.

What you describe has no real impact on data consistency. The only issue is that because the record was not marked in DeltaIndexing yet, it is processed again. So, yes the record is processed twice, which may be a waste of CPU resources but I guess this is acceptable as it only happens occasionally.

A good practice is to stop any crawling processes before shutting down SMILA :-)

Bye,
Daniel

> -----Ursprüngliche Nachricht-----
> Von: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> bounces@xxxxxxxxxxx] Im Auftrag von Andreas.Weber@xxxxxxxxxxx
> Gesendet: Mittwoch, 20. Mai 2009 11:46
> An: smila-dev@xxxxxxxxxxx
> Betreff: [smila-dev] Inconsistent Connectivity state
> 
> Hi,
> 
> I wonder if that is an issue:
> 
> When indexing a datasource, Connectivity does (per record) a routing to
> the queue and
> a visited call to the DeltaIndexing.
> 
> But these two things are not transactional bound.
> 
> When SMILA goes down while indexing, you can come into a situation
> where
> record X
> has been routed to the queue, but is not marked visited in the
> Deltaindexing.
> 
> So, after restarting SMILA and the Crawl-Job, record X will be routed
> to
> the queue again.
> In normal case this will cause a double indexing of the same record.
> 
> What do you think, is that:
> - an issue
> - too esoteric
> - acceptable
> - not implemented yet  ;)
> 
> Best regards,
>  Andreas
> 
> 
> 
> 
> 
> 
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev