Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [m2e-users] Any way to share the repository index between workspaces?

See here for a discussion of Lucene index sharing:
http://www.quora.com/How-can-two-applications-share-the-index-built-by-Lucene

In theory it should be possible to share an index among multiple appplications running in separate JVMs, as long as only one would modify it at any given time. Lucene has no problem with reading and updating an index concurrently. The changes become visible to the readers in transactional fashion. This happens for readers + writers running in the same JVM but AFAIK the atomic update behavior is achieved using filesystem level mechanism so it should work equally well across JVM boundary.

The main problem I see here is maintaining the exclusive write lock. Again, a filesystem level lock would be necessary, and lock breaking / stealing mechanism would be necessary to unwedge updates after the writer process crashes / is killed etc.

Another potential solution is delegating index maintenance and search to an external deamon - it could be Solr (or possibly Nexus?) instance running on the developer's machine or somewhere in the LAN (even more storage / bandwidth savings!)

regards,
Rafał


On 04/25/2012 06:14 PM, Cèsar Ordiñana wrote:
El 25/04/12 17:02, Igor Fedorenko escribió:
No, not currently possible but we welcome quality patches.

Note that we had troubles sharing lucene indexes (this is what m2e uses
internally) among multiple processes in the patch, so I don't know if it
is even possible to reliably share repository indexes among multiple
workspaces.

Also note that m2e keeps at least three copies of each repository index.
The original .gz files downloaded from remote repositories. A shared
lucene instance used to optimize time spent processing gz files. And
per-workspace lucene indexes. The first two are stored under local
repository .cache/m2e directory, the last one is per-workspace.

Ok, I see its not an easy problem to solve.

As I'm on linux, I was thinking as a quick hack to try using symbolic links from all the workspaces to one with the real indexes, but from your information I think that would be a really bad idea.

Thanks for you quick answer Igor!
--
Cèsar


--
Regards,
Igor

On 12-04-25 8:27 AM, Cèsar Ordiñana wrote:
Hi.

I'm working with many related projects, having each one on its own
eclipse workspace. On each of those workspaces, the m2e plugin downloads
the maven repository index into the
".metadata/.plugins/org.eclipse.m2e.core/nexus" workspace folder.

The index files takes around 260Mb of disc space. This is becoming a
real problem for me, as having near 20 workspaces takes around 5Gb of
disc space for all the (repeated) indexes.

Is there any way to configure where the index is stored?

This way it could be configured on each workspace to use a common place,
as it is really always the same index file, saving a lot of disc space,
and also unneeded connections to the main repository server.

The eclipse version I'm using currently is:

Eclipse IDE for Java Developers
Version: Indigo Service Release 2
Build id: 20120216-1857

Regards.
--
Cèsar
_______________________________________________
m2e-users mailing list
m2e-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/m2e-users
_______________________________________________
m2e-users mailing list
m2e-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/m2e-users

_______________________________________________
m2e-users mailing list
m2e-users@xxxxxxxxxxx
https://dev.eclipse.org/mailman/listinfo/m2e-users



Back to the top