AW: [smila-dev] Datasource locking in Deltaindexing Manager

[<Andreas.Weber@xxxxxxxxxxx>]

Hi,

short remark:
When implementing a solution for this, we should also consider (solve) the following problem:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=277083

Cheers,
 Andreas

> -----Ursprüngliche Nachricht-----
> Von: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] Im Auftrag von
> Daniel.Stucky@xxxxxxxxxxx
> Gesendet: Mittwoch, 20. Mai 2009 09:55
> An: smila-dev@xxxxxxxxxxx
> Betreff: AW: [smila-dev] Datasource locking in Deltaindexing Manager
> 
> Hi all,
> 
> a locking mechanism is required to assure that only one client (in this case a CrawlerController
> crawling a specific data source) initializes and executes DeltaIndexing on this datasource.
> DeltaIndexing does not support parallel indexing of the same data source. This could be supported if
> DeltaIndexing-delete is disabeled (just adding/updating records).
> 
> Locking of the Thread id was the easiest solution without changing the interface I think.
> 
> One solution could be to use session IDs. A session ID would be created by calling initialize() and
> would have to passed with all other method calls.
> 
> Bye,
> Daniel
> 
> > -----Ursprüngliche Nachricht-----
> > Von: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-
> > bounces@xxxxxxxxxxx] Im Auftrag von Juergen.Schumacher@xxxxxxxxxxx
> > Gesendet: Mittwoch, 20. Mai 2009 09:37
> > An: smila-dev@xxxxxxxxxxx
> > Betreff: [smila-dev] Datasource locking in Deltaindexing Manager
> >
> > Hi,
> >
> > I'm currently experimenting with the Distributed OSGi (aka RFC 119)
> > implementation of ECF 3.0
> > for exposing SMILA services for remote calls. Specifically I tried to
> > expose the Connectivity
> > Manager service so that we can have Crawlers and Agents talk to a
> > remote service instead of having
> > them always in the same VM. In principal this works quite OK, but I
> > encountered a problem in the
> > Delta Indexing manager: When it initializes a data source for a crawl
> > process, it locks it and
> > associated it to the current thread. This works fine when the crawler
> > is in the same VM because
> > then the complete crawl is done in a single crawl thread. However, with
> > remote access to Connectivity
> > Manger this is not true anymore: servers have usually worker thread
> > pools for executing remote
> > calls and consecutive calls from a single client are not served by the
> > same thread. Of course we
> > could restrict the thread pool size to 1, but I suppose that would kill
> > scalability just a bit (-;
> >
> > Thus, if we really need this data source locking for a crawl thread, I
> > think we need a different
> > method than the current one. Any ideas?
> >
> > Cheers,
> > Jürgen.
> > _______________________________________________
> > smila-dev mailing list
> > smila-dev@xxxxxxxxxxx
> > https://dev.eclipse.org/mailman/listinfo/smila-dev
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev