[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
[
List Home]
|
RE: [smila-dev] Lucene indexing performance
|
For performance reasons it's best to keep the instances of searcher, indexer and reader on Lucene indices. Not to do --> performance penalty.
But I am not sure whether a 80% decrease could be argumented that way.
When working concurrently on those objects (searcher, indexer, ...) the issue is that when parallel changed there may be different views on an index...
I do not currently know the results. Specs and so on has to be read for that purposes. Things I know is that there is a issue with accessing documents in index and positioning things... (get doc with #874534534)
Kind regards,
Georg
-----Original Message-----
From: smila-dev-bounces@xxxxxxxxxxx [mailto:smila-dev-bounces@xxxxxxxxxxx] On Behalf Of Ivan Churkin
Sent: Mittwoch, 13. Mai 2009 14:09
To: Smila project developer mailing list
Subject: Re: [smila-dev] Lucene indexing performance
btw, Lucene IndexWriter supports multiple threads access...
Regards, Ivan
Daniel.Stucky@xxxxxxxxxxx wrote:
> Hi all,
>
> during an index build (over 150.000 documents) we noticed that indexing
> speed gets slower as the index increases in size. Compared to the first
> hour of execution, the 2nd hour was only capable of indexing 80% of the
> load that was indexed in the first hour.
>
> I took a look at the Lucene integration code (by brox) and found, that
> for each index update (add or delete) a new IndexWriter is created and
> closed. This assures that the document is committed for IndexReaders and
> the index is flushed, but I guess that it's bad for performance.
>
> What were the reasons for implementing it that way ? Wouldn't it be
> possible to reuse an IndexWriter, flushing the index either by Memory
> usage or number of documents added/deleted ?
>
> Bye,
> Daniel
> _______________________________________________
> smila-dev mailing list
> smila-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/smila-dev
>
_______________________________________________
smila-dev mailing list
smila-dev@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/smila-dev
