Community
Participate
Working Groups
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729) Build Identifier: 200906160459 CDO tries to use child newer revision, then the father’s revision. This cause object not found exception Reproducible: Always Steps to Reproduce: You will need two sessions with passiveUpdate = false; 1. Open session s1 and transaction t1 load repository. 2. Add object o1 with children’s ch1...ch10 into repository. 3. Commit t1. 4. Open session s2 and transaction t2 load repository. 5. Load object o1 6. In transaction t1 delete child ch1 and commit 7. Iterate children in transaction t2 You will get object not found exception
org.eclipse.emf.cdo.util.ObjectNotFoundException: Object OID13 not found (temporary = false) at org.eclipse.emf.internal.cdo.util.FSMUtil.validate(FSMUtil.java:250) at org.eclipse.emf.internal.cdo.view.CDOViewImpl.createObject(CDOViewImpl.java:824) at org.eclipse.emf.internal.cdo.view.CDOViewImpl.getObject(CDOViewImpl.java:722) at org.eclipse.emf.internal.cdo.transaction.CDOTransactionImpl.getObject(CDOTransactionImpl.java:567) at org.eclipse.emf.internal.cdo.view.CDOViewImpl.convertIDToObject(CDOViewImpl.java:1016) at org.eclipse.emf.internal.cdo.CDOStore.convertToEMF(CDOStore.java:384) at org.eclipse.emf.internal.cdo.CDOStore.get(CDOStore.java:177) at org.eclipse.emf.ecore.impl.EStoreEObjectImpl$BasicEStoreEList.delegateGet(EStoreEObjectImpl.java:225) at org.eclipse.emf.common.util.DelegatingEList.get(DelegatingEList.java:396) at org.eclipse.emf.common.util.DelegatingEList$EIterator.doNext(DelegatingEList.java:1075) at org.eclipse.emf.common.util.DelegatingEList$EIterator.next(DelegatingEList.java:1062) at com.nomagic.cdo.test.tests.Bugzilla_290032_DeleteAndGetObject.testComparisonAcrossTransactions(Bugzilla_290032_DeleteAndGetObject.java:39) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at junit.framework.TestCase.runTest(TestCase.java:164) at junit.framework.TestCase.runBare(TestCase.java:130) at org.eclipse.net4j.tests.AbstractOMTest.runBare(AbstractOMTest.java:109) at junit.framework.TestResult$1.protect(TestResult.java:106) at junit.framework.TestResult.runProtected(TestResult.java:124) at junit.framework.TestResult.run(TestResult.java:109) at junit.framework.TestCase.run(TestCase.java:120) at org.eclipse.net4j.tests.AbstractOMTest.run(AbstractOMTest.java:131) at junit.framework.TestSuite.runTest(TestSuite.java:230) at junit.framework.TestSuite.run(TestSuite.java:225) at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386) at org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(RemotePluginTestRunner.java:62) at org.eclipse.pde.internal.junit.runtime.CoreTestApplication.run(CoreTestApplication.java:23) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.eclipse.equinox.internal.app.EclipseAppContainer.callMethodWithException(EclipseAppContainer.java:574) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:195) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:386) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:585) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:549) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:504) at org.eclipse.equinox.launcher.Main.run(Main.java:1236) at org.eclipse.equinox.launcher.Main.main(Main.java:1212)
Created attachment 147715 [details] TestCase
(In reply to comment #2) > Created an attachment (id=147715) [details] > TestCase This is a problem that the back-end needs to support. If it doesn't ... we cannot do much about it ! It will depends of the implementations of the back-end! Simon
Simon, 1. Which back-ends supporting it? We using back-end with audit support. 2. There is API to load revisions by time (getRevisionByTime). I think in this case to not broke data integrity revision could be loaded by last session refresh timestamp. Could work like audit view when you see all objects at a give time. Regards, Saulius
(In reply to comment #4) > Simon, > > 1. Which back-ends supporting it? We using back-end with audit support. > 2. There is API to load revisions by time (getRevisionByTime). I think in this > case to not broke data integrity revision could be loaded by last session > refresh timestamp. Could work like audit view when you see all objects at a > give time. > > Regards, > Saulius To support it,in passivemode == false, back-end needs to assign a specific db session to a CDOSession on the client. I built Objectivity Store that way. That way, when you access object the back-end it "should" always keep reference to old objects... until it commit or refresh. It should work also with relational database... Simon
Simon, Can you give more details on this solution? Looks like it is not standard CDO code. From you I understand back-end needs to keep old revisions only if there is no audit support, otherwise why we cannot just load from db? I wonder how other people are using CDO and don't get into this problem. Seems like major integrity issue. Maybe only we using session with passiveUpdate=false? :( Saulius
(In reply to comment #6) > Simon, > > Can you give more details on this solution? Looks like it is not standard CDO > code. From you I understand back-end needs to keep old revisions only if there > is no audit support, otherwise why we cannot just load from db? > [SIMON] No, I didn't said it needs to keep old revisions. What I will explain it is not related whatsoever to CDO. Usually relational database and object database have an infrastructure to keep integrity of the table/graph. When you fetch an object ? They can (at different level) keep internally a way to access other objects for that period.. so even if one transaction add new objects or deleted objects, these objects are still accessible if the transaction already did a read on these table... > I wonder how other people are using CDO and don't get into this problem. Seems > like major integrity issue. Maybe only we using session with > passiveUpdate=false? :( [SIMON] Again this isn't a CDO issue, this will rely on the back-end. You will have the same problem in many case ... even in straight EMF. where object into a list belong to different resource... and one of them was deleted.... you Will have the same kind of exceptions. Simon > > Saulius
Simon, This isn't really a bug, because CDO's current behavior is perfectly reasonable. It's more of an enhancement request ;-) (And should probably have been logged as one.) What we would like to see is a read-write view that is safer for the client to read. That is, a R/W view that minimizes (or eliminates) the risk that a *read* operation will throw an exception due to a concurrent change by another client. It seems to us this would not be very hard to implement. For example, a TX could have an option to be "sticky", i.e. load revisions corresponding to the time that the TX was opened (or last refreshed), instead of getting the latest revisions. We realize that with this option turned on, the server will sometimes provide stale revisions to the client; if the client makes changes on such objects and commits them, they will certainly fail -- but that's the client's problem, and a risk he chooses to take if he turns this "sticky" option on. The client can in fact mitigate this risk to a very large extent by doing a refresh when the user actually starts editing. The usefulness of this kind of view comes from the fact that a user typically cannot say in advance whether he will make changes or not. He might just want to browse the data while retaining the option of editing. As long as he's only browsing, we'd like to avoid any read exceptions due to concurrent updates. What do you think? Thanks, Caspar.
If the user is not sure whether he will want to modify objects or not it might be smart enough to expect the "worst" and just start over with a transaction instead of a view? In terms of cost there's not much of a difference between views and transactions as long as nothing is actually modified.
(In reply to comment #9) > If the user is not sure whether he will want to modify objects or not it might > be smart enough to expect the "worst" and just start over with a transaction > instead of a view? In terms of cost there's not much of a difference between > views and transactions as long as nothing is actually modified. We're not worried about the additional cost of opening a tx instead of a read-only view. As you say, we can expect the worst and just use a TX -- but this isn't the issue. Our problem with a regular TX (using lazy loading), is that it cannot guarantee that a read operation will not fail. This is because a TX always fetches the *latest* revisions, which may have been affected by other clients since the last lazy load. So even simple things such as the following, might go wrong: if (list.size() == 10 ) // assume list owner was loaded earlier { Object o = list.get(9); // triggers a lazy load, and therefore // might fail if other client removed object at index #9 } What we're asking for, is a r/w view that eliminates this kind of problem. I.e. a view that guarantees that reads will succeed, by not getting the latest revision, but by getting all revisions for the timestamp at which the view was opened (or last refreshed). As far as read behavior is concerned, this is of course the same as an audit view opened for timestamp = systemtime. But an audit view is not writeable. We want this new view type to be writeable, because often the client won't have any problems if he updates objects and commits -- only if indeed another client beat him to it, will his commit be rejected.
The code example you gave would suffer from the same problem in a purely local, but multi-threaded scenario. There you would add synchronization. In CDO remote scenarios I'd expect explicit locking being used, right? If you just want to prevent the local CDO framework to apply remote invalidations I think CDOView.getLock() should help. Does it?
(In reply to comment #11) > The code example you gave would suffer from the same problem in a purely local, > but multi-threaded scenario. There you would add synchronization. In CDO remote > scenarios I'd expect explicit locking being used, right? With explicit locking it is remote call always. It costs. Plus it blocks writing. Lazy loading is also remote call, but only when cache missing it. Even in such case, loading one revision does not block commits. Another problematic place is generic code reading EMF code and not aware of CDO. Such code could work without additional CDO read locks. Even if we go with CDO read locks, there might be cases when I start reading model v1(time t1) and end with v2(time t2). For example I read root object recursively. For the beginning I can lock root object only. Get children for root and lock each child. While I reading root children, one of children could be changed (v2 committed), because children are locked later. So currently we can only use audit view to read model consistently. It works, but I would prefer to read and write in the same view (one transaction). > > If you just want to prevent the local CDO framework to apply remote > invalidations I think CDOView.getLock() should help. Does it? This particular issue we experience with passiveUpdate=false, so remote invalidation isn't a problem. BTW, does remote invalidation applies changes in transactional style also? Like: 1. lock view 2. apply all one commit changes 3. unlock view Saulius
> BTW, does remote invalidation applies changes in > transactional style also? > > Like: > 1. lock view > 2. apply all one commit changes > 3. unlock view Yes, see CDOViewImpl.handleInvalidation(): lock.lock(); try { conflicts = handleInvalidationWithoutNotification(dirtyOIDs, detachedOIDs, dirtyObjects, detachedObjects); } finally { lock.unlock(); } But the sole purpose of this lock is that a user can use it explicitely to suspend invalidation handling. It's not used by the framework otherwise.
I just noticed that your last question could imply a misunderstanding. When handling remote invalidations the framework will never apply changes other then marking CDOObjects PROXY or CONFLICT (plus setting the CDOObject's revision pointer to null in case of PROXY). IIRC even in the case of a change subscription the modeled state of this object is not changed. The delta is only used to emit EMF style notifications. Sijmon, do you think we could use change deltas to construct a local revision copy of the latest state? Then a subsequent LoadTransition would not cause a server round trip, but find the needed revision in the local cache...
(In reply to comment #14) > I just noticed that your last question could imply a misunderstanding. When > handling remote invalidations the framework will never apply changes other then > marking CDOObjects PROXY or CONFLICT (plus setting the CDOObject's revision > pointer to null in case of PROXY). > > IIRC even in the case of a change subscription the modeled state of this object > is not changed. The delta is only used to emit EMF style notifications. > > Sijmon, do you think we could use change deltas to construct a local revision > copy of the latest state? Then a subsequent LoadTransition would not cause a > server round trip, but find the needed revision in the local cache... We could if the version received == current version of the view + 1 Simon
Just to be clear.. By doing that.... it will not mean that this bug will be solved. You will still encounter that problem(this bug).
Yes, I understand. It's more about an additional internal optimization. Thx ;-)
Rebasing all outstanding 3.0 problem reports to version 3.0.1
Fixing wrong bug version.
I'd like to put this back on the table for 4.0. Recap: the idea is to have CDO transactions (i.e. R/W views) of a special type, that we could call "time-pegged". That is to say, they're pegged to the timestamp at which they're opened, until an explicit refresh. Such a tx would load revisions for that timestamp, not the latest revisions as a normal tx would. I have a working (or so it seems) implementation of this in my custom codebase. Will try to create a 4.0 patch.
Created attachment 179230 [details] Patch including testcase
Created attachment 179430 [details] Patch v2 Uploading an enhanced version of the patch which, during postcommit, tracks for every committed object onto which branchPoint it was committed, so that the correct revision will be fetched when a sticky TX must reload an object that it itself committed earlier. On Skype we also talked about eliminating setPassiveUpdateEnabled in favor of setPassiveUpdateMode(NONE), as part of this patch. I didn't go ahead with this, as it impacts the GUI, which is unknown territory for me.
Created attachment 179502 [details] Patch v3 Patch amended again, in accordance with what we agreed to on Skype: sticky behavior shall be automatic when passiveUpdates==false and the IStore supports auditing. Also fixed a few failures in tests that assumed non-sticky behavior.
Created attachment 180029 [details] Patch v4 - ready to be committed
Committed to HEAD
Available in R20110608-1407
