[smila-dev] Datasource locking in Deltaindexing Manager

[<Juergen.Schumacher@xxxxxxxxxxx>]

Hi,

I'm currently experimenting with the Distributed OSGi (aka RFC 119) implementation of ECF 3.0
for exposing SMILA services for remote calls. Specifically I tried to expose the Connectivity
Manager service so that we can have Crawlers and Agents talk to a remote service instead of having
them always in the same VM. In principal this works quite OK, but I encountered a problem in the
Delta Indexing manager: When it initializes a data source for a crawl process, it locks it and
associated it to the current thread. This works fine when the crawler is in the same VM because 
then the complete crawl is done in a single crawl thread. However, with remote access to Connectivity
Manger this is not true anymore: servers have usually worker thread pools for executing remote 
calls and consecutive calls from a single client are not served by the same thread. Of course we 
could restrict the thread pool size to 1, but I suppose that would kill scalability just a bit (-;

Thus, if we really need this data source locking for a crawl thread, I think we need a different 
method than the current one. Any ideas?

Cheers,
Jürgen.