Community
Participate
Working Groups
Build Identifier: I don't know what is the status of the EMF/Sphinx new indexing implementation but the current implementation, as you know, is quite slow. In my project, that is made from 1700+ files and more then 600k EObjects the EcoreIndex.init is responsible for about 32% of the loading time. On the other hand there are models that can have a much faster implementation. For example, the URI of an EObject can be computed much faster if you can have already the URI of the owner object. Which EcoreIndex already have... I propose to make the EcoreIndex non-final. Even better, create an interface and transform the current class in a default implementation. Then an implementer can provide an adapter inside the ResourceSet for the ProxyHelper Because it is planned to drop the current index in the future the implementation shall be put in a provisional state as per http://wiki.eclipse.org/Provisional_API_Guidelines Reproducible: Always
(In reply to comment #0) > I don't know what is the status of the EMF/Sphinx new indexing implementation We have made some experiments using the indexing infrastructure of Xtext. We took a 40MB XML size real world AUTOSAR model and indexed all identifiable objects. Result: the index size in memory was approx. 200MB... But even if the extra memory space required for the index were - in an ideal world - zero, the benefit of an index would be somehow limited. In would accelerate proxy resolving and enable us to switch from eager loading to lazy loading. However, after some time, all proxies are resolved and all files are loaded. Then the index is of no more advantage and the real problem comes into play: memory space required by the whole model and the fact that you have not chance to get rid of model parts which you no longer need, because all references between model objects are strong Java references. The most promising way to get over this and enable unneeded parts of a model to be garbage collected is leveraging model repository technologies like CDO. It has a dedicated memory management capability which does exactly what we need here. Even if we want to continue to work with files and don't plan to put our AUTOSAR models directly in a model repository on a big server, we could still use CDO as a local cache and leverage these memory management capabilities. The model indexing story looks however not very promising and I'm not sure if it makes sense to pursue this path. > but the current implementation, as you know, is quite slow. In my project, that > is made from 1700+ files and more then 600k EObjects the EcoreIndex.init is > responsible for about 32% of the loading time. In our test models it was less than that, but if you have many cross references it can easily go up to such values. But note that the proxy resolution in models with fragment-based URIs, such as AUTOSAR, would take even much more time, if we hadn't the EcoreIndex at all. > On the other hand there are models that can have a much faster implementation. > For example, the URI of an EObject can be computed much faster if you can have > already the URI of the owner object. Which EcoreIndex already have... Ok, I see, you mean that the EcoreIndex initialization could be optimized by replacing the generic Resource#getURIFragment(o) calls by an optimized metamodel-specific algorithm. > I propose to make the EcoreIndex non-final. Even better, create an interface and > transform the current class in a default implementation. Then an implementer can > provide an adapter inside the ResourceSet for the ProxyHelper I agree. And the optimized implementation for AUTOSAR which you have sketched above should be added to Artop then. > Because it is planned to drop the current index in the future the implementation > shall be put in a provisional state as per > http://wiki.eclipse.org/Provisional_API_Guidelines For the reasons that I have explained above, it could happen that we finally keep the EcoreIndex. But it will need to be refactored, so the current state clearly is provisional no matter what will happen. That said, I agree that API guidelines should be followed and even other must-dos as per http://eclipse.org/indigo/planning/EclipseSimultaneousRelease.php should be addressed. But it should be done globally, not only in the present case. I'm currently checking if I'll get a chance to work on this in the coming months. Joining the Juno release train with Sphinx next year would be a nice thing!
I might have time to work on this in the July/August. Also on #349655 and #349675
I've made some changes and attached a patch with the changes performed. The changes do not have the scope to improve the performance of the existing implementation, but to provide a more extensible framework that will allow the adopters to benefit from the particularities of the used meta-model and provide a more efficient algorithm. In this scope, some interfaces has been introduced (for the existing ECoreIndex and ModelIndex classes) together with some needed refactoring.
Created attachment 232445 [details] Proposed patch for providing a more extensible framework.
Closed stale issue before migration
