Community
Participate
Working Groups
Using CDO with dynamic packages is very error prone when one goes beyond the basic scenario of having a single package with no dependencies. Once one attempts to work with multiple interdependent packages, out-of-the-box there are various problems: - Ecore XML sent to the server contains file references instead of nsURI references, breaking dependency resolution on the server. - Failure to resolve package dependencies on the client, when packages are first used and sent to the server, gives no error message. - Failure to resolve packages dependencies on the server, when packages are used in later transactions, also gives no error message. One can work around these problems by patching the Ecore files, using URI mappings, forcing dependency resolution etc., but it's all rather uninviting and very hard to debug when anything goes wrong. For example, when the client can resolve a package dependency but the server can not, this goes undetected until there is actual transmission of data, when the mismatch between the server's reading logic and the client's writing logic will (probably) lead to some low-level error. I believe we should invest in adding some checks to fail early with clear error messages, and maybe add some mechanisms to relieve the app coder of having to think a lot about the subtle differences between generated packages and dynamic packages.
I fully agree.
Can someone have a look at Bug 302838 and tell me if this could also be the problem there?
(In reply to comment #2) > Can someone have a look at Bug 302838 and tell me if this could also be the > problem there? See my comment in bug 302838.
Any news here?
(In reply to comment #4) > Any news here? Not in the way of solutions for the various symptoms of this issue. But I have a curious one to add to the list: whether or not the server adequately processes a set of interdependent dynamic packages when they are first committed, seems in certain cases to depend on the order in which they are received. This order is not deterministic, probably due to non-deterministic allocation of the packages in a client-side HashMap. Will investigate further.
An update: at long last I've discovered why the server's ability to correctly process incoming dynamic packages, depends on the order in which they arrive (which is non-deterministic). During construction of EPackage P1's content from the incoming XML, references to other packages may or may not be resolvable, depending on whether that referenced package (P2) was already received and processed or not. This in itself is not a problem (because resolution is taken care of later), but when the referenced package is NOT resolvable, a resource is nevertheless created for it. This resource fails to load, but is marked as loaded anyway in the ResourceSet (this seems to be by-design EMF behavior). Later on, when the XML for P2 is itself processed, CDO explicitly creates a new resource for it, even though an empty resource for it may already have been created "implicitly", as just described. So that gives 2 resources designated by P2's URI, the 2nd one actually containing the EPackage content. But the 1st one will get "consulted" when an reference owned by an element in P1 must be resolved. This resolution always fails because that 1st resource is simply empty, but it fails silently (again, apparently this is by EMF design). Ultimately the problem manifests itself as communication problems when CDORevisions are committed, due to the server having a different idea of, for example, the number of features that some EClass has, the server having misplaced features inherited from a base class contained in a different package. It took me ages to debug this because every aspect of this problem goes undetected, i.e. the failure to load the referenced resource/package, the failure to resolve during construction of the ePackage, the creation of a second resource with the same URI, and the failure to resolve at a later stage. I have a patch in place on my own custom code base, which seems to fix the problem. Will attempt to rework it for 3.0.
That indeed sounds scary. I'm looking forward to look at the patch that you mentioned. That will make it clearer to me what really goes on here. Thanks for not giving up! Do you think Ed should be involved to comment on EMF-related issues?
Created attachment 164530 [details] Patch This should fix the problem of duplicate resources getting implicitly created during the processing of new packages, with the empty "duds" nevertheless being referenced by some dependent packages. (The solution is in EMFUtil.createEPackage.) With this patch in place, things will at least work correctly if all interdependent packages are transmitted. There's still no detection of a missing package though.. Will work on that.
Created attachment 164545 [details] Patch v2 - ready to be committed
(In reply to comment #9) Committed to HEAD. Not resolving bug, because this patch fixed only one of several issues.
Created attachment 165245 [details] Testcase (as a patch) This patch contains two testcases, one demonstrating how CDO does not detect a missing dependency when a new package is committed; and the second demonstrating how it does not detect that the dependent package references the dependency through a file URI -- which cannot possibly be resolved on the server side even if the required package is sent.
Working on a patch to detect missing dependencies. EMF doesn't seem to provide any tools to perform the proxy resolution in a way that's quite right for the situation at hand.. having a discussion with Ed about this, here: http://www.eclipse.org/forums/index.php?t=tree&th=166582
Created attachment 165723 [details] Patch including testcase, for 3.0
Created attachment 166162 [details] Ready to be committed
I noticed that your 2 tests do not properly run if they're called from other plugins (like the DBStore tests). Can you somehow use something like OM.BUNDLE.getInputStream(fileName) to load the model? There's an example in org.eclipse.emf.cdo.tests.PackageRegistryTest.loadModel(String). Maybe you want to make this static helper method public and just reuse it?
Tests fixed as you suggested: by making PackageRegistryTest.loadModel public and reusing it. Committed to HEAD.
Actually I just realized that it would make more sense to move the loadModel(String) method to EMFUtil. Shall I move it?
Then we would have to pass in the OMBundle instance. Otherwise the common plugin would not find the file in the test plugin. I think that doesn't rectify the effort. But thanks for being sensible ;-)
Available in 3.0 GA: http://download.eclipse.org/modeling/emf/cdo/updates/3.0-releases/
