Bug 233495 - Implement a new EMAP to have better performance for big collection
Summary: Implement a new EMAP to have better performance for big collection
Status: NEW
Alias: None
Product: EMF
Classification: Modeling
Component: cdo.core (show other bugs)
Version: 4.13   Edit
Hardware: PC Windows XP
: P1 enhancement with 3 votes (vote)
Target Milestone: Galileo   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard: Lighter, Faster and Better
Keywords:
Depends on:
Blocks: 250036
  Show dependency tree
 
Reported: 2008-05-22 13:06 EDT by Simon Mc Duff CLA
Modified: 2020-12-11 14:49 EST (History)
7 users (show)

See Also:
smcduff: galileo+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Mc Duff CLA 2008-05-22 13:06:43 EDT
Hi Simon,

I suggest that you enter a feature request to discuss the details there.
Please describe where you intend to create instances of such a new EMap.

Cheers
/Eike


Simon McDuff schrieb:
> OK.. basically to provide our own implementation of EMap !!
>
> So we can handle any cases!!
>
>
>
>
> "Eike Stepper" <stepper@sympedia.de> a écrit dans le message de news: 
> g145fg$mtc$1@build.eclipse.org...
>   
>> Simon McDuff schrieb:
>>     
>>> (I`m just describing the current implementation)
>>>
>>> But by knowing that,
>>>   Estore doesn`t to know anything about EMap. Doesn`t need to since 
>>> EBasicMap will only sent the following message to the Estore (not 
>>> directly). Lookup calls are kept internally, so the Estore never know 
>>> about it.. since EBasicMap built is hashmap at the initialization.
>>>
>>> Object get(InternalEObject object, EStructuralFeature feature, int 
>>> index);
>>>
>>> Object set(InternalEObject object, EStructuralFeature feature, int index, 
>>> Object value);
>>>
>>> int size(InternalEObject object, EStructuralFeature feature);
>>>
>>> etc....
>>>
>>>       
>> Then I wonder how a (CDO-based) solution might look like that solves the 
>> problem you initially described?
>>
>>
>> Cheers
>> /Eike
>>
>>     
>>> "Eike Stepper" <stepper@sympedia.de> a écrit dans le message de news: 
>>> g13mpg$1pa$4@build.eclipse.org...
>>>
>>>       
>>>> Simon McDuff schrieb:
>>>>
>>>>         
>>>>> EMAP is a list of a class that only have a key and a value.
>>>>>
>>>>> EMap always add at the end of the list.
>>>>>
>>>>> The estore only knows index of a list!!
>>>>>
>>>>>
>>>>>           
>>>> That's known. But what about the important part: EStore?
>>>>
>>>>
>>>>         
>>>>> "Eike Stepper" <stepper@sympedia.de> a écrit dans le message de news: 
>>>>> g12v36$ik7$1@build.eclipse.org...
>>>>>
>>>>>
>>>>>           
>>>>>> Simon,
>>>>>>
>>>>>> Conceptually I think both approaches can be useful and I know a third 
>>>>>> one which is like a combination of the existing HashMap based 
>>>>>> implementation and your remote lookup. We could first look into the 
>>>>>> HashMap and do the remote lookup (plus caching in the map) only if the 
>>>>>> key is not present. With all three apporaches we'd have to evaluate 
>>>>>> how the other EMap API (other than lookup) fits into the approach.
>>>>>>
>>>>>> Btw. I never used an EMap but I know that EMap extends EList and I 
>>>>>> thought EMap is implemented by subclassing an EList implementation 
>>>>>> plus linear scanning for the lookup. Ok, I just verified that my last 
>>>>>> assumption was wrong (does it apply to EFeatureMaps?). BasicEMap is a 
>>>>>> subclass of java.lang.Object but delegates Entry storage to an EList 
>>>>>> which uses a BasicEList[] (is it clever to have both delegations in 
>>>>>> one EMap implementation, how to implement a different delegateEList 
>>>>>> without the existing entryData?).
>>>>>>
>>>>>> From a CDO client point of view I wonder how an EStore "gets a sense" 
>>>>>> for an EMap typed attribute? I can't find the lookup API in EStore.
>>>>>>
>>>>>> Cheers
>>>>>> /Eike
>>>>>>
>>>>>>
>>>>>> Simon McDuff schrieb:
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Hi Eike and Ed (Ed, you are more than welcome to add anything to this 
>>>>>>> thread),
>>>>>>>
>>>>>>> the current implementation of Emap is using a hashmap. Unfortunately, 
>>>>>>> to use a hashmap we need to load all the elements of the collection. 
>>>>>>> This kill the point of proxying objects (when we are using EStore). 
>>>>>>> We have cases where a collection reach millions of objects.... I 
>>>>>>> don`t want to load all of them.
>>>>>>>
>>>>>>> I tought that an implementation using TreeMap would be better (It is 
>>>>>>> an ordered collection). In this case, I can do dichotomic search... 
>>>>>>> so I will not load all of them!
>>>>>>> Do you think it make sense ?
>>>>>>>
>>>>>>> The other solution I can think of its to catch the lookup and always 
>>>>>>> go to the server to lookup objects. In the server side , IStoreReader 
>>>>>>> will need to provide some functionnality to handle this 
>>>>>>> (lookup(CDOFeature, Object key) return index).
>>>>>>>
>>>>>>> Maybe have both?!?!?!?
>>>>>>>
>>>>>>> Do you have others ideas to solve that kind of problem ?
>>>>>>>
>>>>>>> Thank you
>>>>>>>
>>>>>>> Simon
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>           
>>>
>>>       
>
>
>
Comment 1 Eike Stepper CLA 2008-06-10 02:30:28 EDT
Reversioned due to graduation
Comment 2 Eike Stepper CLA 2009-11-01 05:58:15 EST
Rebasing all unresolved enhancement requests to 3.0
Comment 3 Eike Stepper CLA 2010-06-29 04:50:01 EDT
Rebasing all outstanding enhancements requests to version 4.0
Comment 4 Jon Sorensen CLA 2011-03-10 14:01:06 EST
Not to come across as harsh or ungrateful but this seems like basic functionality to me.

One of the primary reasons our application uses CDO is to leverage the scalability of a database for relatively large amounts of data. Having an EMap implementation that pulls everything into memory negates the benefits of the underlying database.
Comment 5 Jon Sorensen CLA 2011-03-10 14:02:12 EST
Not to come across as harsh or ungrateful but this seems like basic functionality to me.

One of the primary reasons our application uses CDO is to leverage the scalability of a database for relatively large amounts of data. Having an EMap implementation that pulls everything into memory negates the benefits of the underlying database.
Comment 6 Eike Stepper CLA 2011-03-10 17:37:03 EST
Yeah, this feature would definitely suit CDO very well ;-)
Comment 7 chengdong Mising name CLA 2011-03-10 21:10:29 EST
+1 for this feature.
Comment 8 Eike Stepper CLA 2011-06-23 03:56:51 EDT
Moving all open enhancement requests to 4.1
Comment 9 ken griffey CLA 2011-07-20 03:10:07 EDT
+1 for this feature.
www.nikejrshoes.com
Comment 10 Eike Stepper CLA 2012-08-14 22:54:59 EDT
Moving all open issues to 4.2. Open bugs can be ported to 4.1 maintenance after they've been fixed in master.
Comment 11 Eike Stepper CLA 2013-06-27 04:05:44 EDT
Moving all outstanding enhancements to 4.3
Comment 12 Eike Stepper CLA 2014-08-19 09:22:38 EDT
Moving all open enhancement requests to 4.4
Comment 13 Eike Stepper CLA 2014-08-19 09:34:47 EDT
Moving all open enhancement requests to 4.4
Comment 14 Ed Willink CLA 2014-09-01 11:13:42 EDT
This seems like an EMF Core as much as a CDO issue; see also Bug 443021.
Comment 15 Eike Stepper CLA 2015-07-14 02:17:45 EDT
Moving all open bugzillas to 4.5.
Comment 16 Eike Stepper CLA 2016-07-31 01:00:38 EDT
Moving all unaddressed bugzillas to 4.6.
Comment 17 Eike Stepper CLA 2017-12-28 01:09:19 EST
Moving all open bugs to 4.7
Comment 18 Eike Stepper CLA 2019-11-08 02:03:58 EST
Moving all unresolved issues to version 4.8-
Comment 19 Eike Stepper CLA 2019-12-13 12:45:17 EST
Moving all unresolved issues to version 4.9
Comment 20 Eike Stepper CLA 2020-12-11 10:39:07 EST
Moving to 4.13.
Comment 21 Ed Willink CLA 2020-12-11 14:49:30 EST
Surely this should be an EMF Core facility? Currently EMap is well-suited to XML serialization since it is really just an EList of key-value pairs. However its functionality is very disappointing compared to a Map which is not even an inherited EMap interface.

See Bug 443195 where I tried to enhance the support of eKeys to correspond to a UML map but failed. At this point, I doubt there is a solution that does not run headlong into the problem of legacy compatibility.

(Eclipse OCL introduced a Map to its standard library a few years ago so the inefficiency of EMap is mitigated as a one-off serialization intermediate between the XML list-of-pairs and the more efficient OCL computational element. EMF supports use of OCL standard library types, which are just Java types, in the same way as any other Java type. This kind of approach might provide a solution for some CDO users.)