[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [rdf4j-dev] Newer Lucene Version

Hi Bart,

thanks for the feedback.

I will go for the intermediate update then and try to provide a change.

For the lucene upgrade it is pretty much straight forward (lucene 6 and lucene 7 are mostly compatible on the API level), i.e. I could almost reuse my change from the evaluation. Once the code change is complete I will test the upgrade in one of our installations and check the compatibility of lucene indices.

Elasticsearch unfortunately is a different story with quite a lot of API changes and no clear migration guide. I will try to resolve the required changes and then someone with good knowledge about elasticsearch should do a careful review. So far I have not worked with elasticsearch at all.

By the way: is there good coverage for these features in the unit tests?

Best,
ÂAndreas



2018-05-16 23:26 GMT+02:00 Bart Hanssens (BOSA) <bart.hanssens@xxxxxxxxxxxx>:

Hi Andreas,

Â

Â

Thanks for the research, IMHO an upgrade would indeed be useful.

Â

And indeed, Elasticsearch 2.4 is EOL as well (https://www.elastic.co/support/eol)

There was at least one similar issue on elasticsearch a few months ago (https://github.com/eclipse/rdf4j-storage/issues/18),

and some discussion on this list on Lucene and Elasticsearch versions (https://dev.eclipse.org/mhonarc/lists//rdf4j-dev/msg00400.html)

Â

Â

Any experience in upgrading existing Lucene 5 installations to 7.x ?

If I read this correctly, one should reindex the data, and skipping a major version does not seem advisable

https://lucene.apache.org/solr/guide/7_0/major-changes-in-solr-7.html

Â

âRe-indexing your data is considered the best practice and you should try to do so if possible.

However, if re-indexing is not feasible, keep in mind you can only upgrade one major version at a time.

Thus, Solr 6.x indexes will be compatible with Solr 7 but Solr 5.x indexes will not be.â

Â

Â

Maybe a two-step approach would be better ?

I.e. first upgrade to Lucene 6.6 / ÂElasticSearch 5.6 (supported until March 2019), then to 7.x in another RDF4J release ?

Â

Â

Best regards

Â

Bart

Â

From: rdf4j-dev-bounces@xxxxxxxxxxx [mailto:rdf4j-dev-bounces@eclipse.org] On Behalf Of Andreas Schwarte
Sent: woensdag 16 mei 2018 11:36
To: rdf4j developer discussions <rdf4j-dev@xxxxxxxxxxx>
Subject: [rdf4j-dev] Newer Lucene Version

Â

Hi,

in our new company we have more strict security guidelines w.r.t open source software.

The lucene version that is currently bundled with RDF4J (i.e. lucene 5.x) is EOL, same for solr (and probably elasticsearch). The current stable is 7.3.0 (or 7.3.1).

How are the policies with an upgrade of these 3rd party components? Could this be done in a 2.4.0 release?

I have done an evaluation of the update. Quite a bit of Lucene API replacements required, but looks pretty dsave. The only thing that I could not solve so far is the update of "elasticsearch", which fails in maven with a "bytecode enforce check" on log4j

[INFO] Restricted to JDK 1.8 yet org.apache.logging.log4j:log4j-api:jar:2.9.1:compile contains META-INF/versions/9/org/apache/logging/log4j/util/ProcessIdUtil.class targeted to JDK 1.9

Any ideas on how to fix this?

If we are ok to get the work on an upgrade started I could upload my change as a pull request.

Opinions?

Thanks,

ÂAndreas


_______________________________________________
rdf4j-dev mailing list
rdf4j-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/rdf4j-dev