Community
Participate
Working Groups
https://www.eclipse.org/forums/index.php/mv/msg/1106617/1836867/#msg_1836867 describes a user's metamodel schizophrenia problem whereby QVTo reports type-is-not-a-type since the genmodel contributes to inconsistent metamodel loading that is hard to avoid. There are three possible design policies: a) (current) users must magically contrive to use the same form of metamodel access everywhere. This is close to impossible given the diversity of tools and policies and switching between development/installed accesses. b) (OCL Pivot) an enhanced metamodel loading policy ensures that all same-nsURI EPackages have only a single copy in memory. See StandaloneProjectMap. c) all tooling code tolerates the schizophrenia so that all comparisons for same type use hierarchical name rather than address equality. ----- c) requires all tooling to be reviewed, so not only all QVTo code but all re-used OCL and possibly even EMF code may need overriding. Fixing equality is tractable, but once the tolerance extends to hierarchical is-a, the comparisons require much more care. And of course if multiple loads are accommodated what properties does C have if one copy provides C::a1 and another provides C::a2? This approach seems doomed to create as many problems as it cures. Much better to cure the problem rather than fudge workarounds to the consequences of not solving the original problem. ----- b) Many years of unifying diverse models for Pivot OCL/QVTd JUnit tests has shown that this approach works, but can require considerable care in setting up the classpath to ensure that metamodels are found and to ensure that resolution of schizophrenic conflicts goes in the correct direction. I sometimes wonder whether the many challenges that StandaloneProjectMap can pose are worse than the problem. But... ----- a) Is not tractable, so for my OCL QVTo automations I achieve the a) solution by wrapping a b) solution around it in an invoking MWE2 script. ------------ Conclusion: Today, users must wrap an external b) around a). Tomorrow, QVTo could move to an internal b) like QVTd. This would be much easier if QVTo was using Pivot rather than Classic OCL.
See https://bugs.eclipse.org/bugs/show_bug.cgi?id=567161#c8 for a more comprehensive variant of b) that distinguishes the pre-gemodel, genmodel, post-genmodel, inverse-genmodel use cases. The QVTo compiler is a pre-genmodel application. The QVTo compiler is a genmodel application. The QVTo execution/runtime is a post-genmodel application.
(In reply to Ed Willink from comment #0) > Tomorrow, QVTo could move to an internal b) like QVTd. This would be much > easier if QVTo was using Pivot rather than Classic OCL. Rubbish. Although StandaloneProjectMap is packaged in the org.eclipse.ocl.pivot plugin, it is just a utility used by the Pivot. It could be factored out to org.eclipse.ocl.{common/util} or cloned by QVTo if this is the way that QVTo wants to move forward.
QVTo once had EPackageRegistryBasedURIResourceMap. Maybe it had a similar intention? https://git.eclipse.org/c/mmt/org.eclipse.qvto.git/tree/plugins/org.eclipse.m2m.qvt.oml/src/org/eclipse/m2m/internal/qvt/oml/compiler/EPackageRegistryBasedURIResourceMap.java The class is no longer used by QVTo because there were no test cases or other known use cases relying on it. Maybe it's not a good idea to just reintegrate the class, but the original integration might provide some indication as to how StandaloneProjectMap would have to be used.
(In reply to Christopher Gerking from comment #3) > QVTo once had EPackageRegistryBasedURIResourceMap. Attempting to understand it. It appears to detect that it is loading e.g. <ecore:EPackage xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:ecore="http://www.eclipse.org/emf/2002/Ecore" name="ecore" nsURI="http://www.eclipse.org/emf/2002/Ecore" nsPrefix="ecore"> where the ecore xmlns is http://www.eclipse.org/emf/2002/Ecore i.e if it is doing a dynamic load of platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore. If this dynamic load fails, a URI mapping is installed from platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore to http://www.eclipse.org/emf/2002/Ecore. It used to be installed by QvtEngine private void reset(QvtCompilerOptions options) { // TODO: QvtException ResourceSetImpl resourceSet = new ResourceSetImpl(); resourceSet.setURIResourceMap(new EPackageRegistryBasedURIResourceMap(resourceSet.getURIConverter())); It appears to have got lost as QvtEngine got refactored. ----- It was a very narrow solution to platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore for everybody always. OCL's StandaloneProjectMap or even EcorePlugin's ExtensionProcessor is a solution for far more but not currently in use. Therefore as a first solution attempt to use ExtensionProcessor whenever a ResourceSet is created. ---- And while configuring a ResourceSEt properly you could include the following which fixes a really obscure bug. /** * The default ResourceSet.getLoadOptions() do not support loading models that reference themselves. * Setting XMLResource.OPTION_DEFER_IDREF_RESOLUTION to true avoids this problem. * See Bug 499442. * @since 1.3 */ public static void initializeLoadOptionsToSupportSelfReferences(@NonNull ResourceSet resourceSet) { resourceSet.getLoadOptions().put(XMLResource.OPTION_DEFER_IDREF_RESOLUTION, true); } ---- QVTo actually has two distinct problems. a) when loading we want to ensure that synonym models are unified. Naively QVTo as an M2M tool should only ever use dynamic models. But this imposes a performance penalty whenever the user has generated Java variants; therefore the fortuitous availability of Java models should make zero difference other than speed. b) when saving the result the user may need compatibility with other tools so use of nsURI or platform: could need to be a per-metamodel user preference. NB. Models used by QVTo tooling must be 100% hidden from the user to ensure that a QVTo M2M used in the development of QVT 2.x does not get infected with the QVT 1.x built-in to QVTo; Acceleo gets this very wrong. ---- Probably a JUnit for a derived genmodel will demonstrate the bug and its fix.
(In reply to Ed Willink from comment #4) > It appears to detect that it is loading e.g. > > <ecore:EPackage xmi:version="2.0" xmlns:xmi="http://www.omg.org/XMI" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > xmlns:ecore="http://www.eclipse.org/emf/2002/Ecore" name="ecore" > nsURI="http://www.eclipse.org/emf/2002/Ecore" nsPrefix="ecore"> In particular it takes effect when using EPackage as a type inside another external metamodel: <eStructuralFeatures xsi:type="ecore:EReference" name="p" eType="ecore:EClass platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore#//EPackage"/> > If this dynamic load fails, a URI mapping is installed from > platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore to > http://www.eclipse.org/emf/2002/Ecore. Not only if the load fails. The SAXException represents the case that ecore::EPackage is loaded. If so, it prevents the reload by reusing ecore from the global package registry. > It used to be installed by QvtEngine ... and other places. > It appears to have got lost as QvtEngine got refactored. Yes. The right to exist was unclear as there was no test case relying on it. And still it wouldn't have solved the user problem from the forum because it only handles ecore::EPackage, not pivot::State or any other custom classes.
(In reply to Christopher Gerking from comment #5) > <eStructuralFeatures xsi:type="ecore:EReference" name="p" > eType="ecore:EClass > platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore#//EPackage"/> This seems a bad practice that we should not encourage and do not need to support. It specifically requests a dynamic load of the installed Ecore, avoiding using of the compiled Java version, but guaranteeing a conflict between dynamic/static Ecore. However I see the UML.ecore, OCLEcore.ecore and OCLUML.ecore do his prolifically, no doubt to coexist with the unavoidable dynamic load by usedGenPackages="platform:/plugin/org.eclipse.emf.ecore/model/Ecore.genmodel#//ecore platform:/plugin/org.eclipse.uml2.types/model/Types.genmodel#//types" I guess a part of the problem is that generation of the compiled Java Ecore model does not also generate a compiled Java GenModel. It would be possible for a revamped EPackageRegistryBasedURIResourceMap to ensure that platform:/plugin/*.ecore maps to its Java and that platform:/plugin/*.genmodel maps to a reference-corrected form of the genmodel. However the real trouble occurs for a multi-level development such as QVTd->OCL->Ecore where Ecore is plugged-in, OCL is under development and QVTd is under development. References from QVTd to OCL must resolve to the project not plugin version requiring the harder mapping that only StandaloneProjectMap does to map the nsURI to the dynamic load occluding the static version. And if for experimentation purposes Ecore is also checked out as a project the references should adjust without editing to hide the Java EPackage. ----- But idealism is too hard. Practically and to avoid imposing the costly classpath search on existing users unnecessarily, we can re-instate EPackageRegistryBasedURIResourceMap to eagerly map platform:/plugin/.../Ecore.ecore to its nsURI and lazily map any platform:/plugin load failure via the EcorePlugin.ExtensionProcessor computations.
(In reply to Ed Willink from comment #6) > This seems a bad practice that we should not encourage and do not need to > support. What practice is actually better? The default practice resulting from "Load resource..." seems to be ../../org.eclipse.emf.ecore/model/Ecore.ecore#//EPackage. To what extent is that a better practice, doesn't it have the same implications? > It would be possible for a revamped EPackageRegistryBasedURIResourceMap to > ensure that platform:/plugin/*.ecore maps to its Java and that > platform:/plugin/*.genmodel maps to a reference-corrected form of the > genmodel. How should the reference-corrected form of the genmodel look like?
(In reply to Christopher Gerking from comment #7) > > It would be possible for a revamped EPackageRegistryBasedURIResourceMap to > > ensure that platform:/plugin/*.ecore maps to its Java and that > > platform:/plugin/*.genmodel maps to a reference-corrected form of the > > genmodel. > How should the reference-corrected form of the genmodel look like? Since post-genmodel, all tooling models are Java generated, a post-genmodel genmodel would only reference Java models. i.e. no platform:/.... always a typically http: nsURI. > What practice is actually better? Very hard to say. Unfortunately EMF gave us a bum steer with potentially three alternative models to choose from and inconsistent referencing and no ability to refer to a workspace model by its nsURI. Downstream from EMF each project tool has its diverse assistance. UML models are unhelpful, Classic OCL referencing UML and Ecore are worse. In a 'simple' world we would always load platform:/resource/... so that the prevailing development version is used falling back to platform:/plugin/... . Tooling may fall back to the nsURI instead giving improved performance. Whatever, there is only one model to load. To achieve this transitively, if OCL loads UML from a plugin, but Ecore is checked out in the workspace, then the workspace Ecore.ecore must be used and so the Java UML is not applicable, rather platform:/plugin UML must be used and its ../.. references to Ecore must hit platform:/resource. In the real world, we have a hotchpotch of references that are very hard for multi-project users to get consistent given that examples encourage strange usage and diverse tooling requires / generates inconsistent style. IMHO platform:/resource is always right, it should falll back. IMHO platform:/plugin is always wrong since it may easily result in the current workspace version being ignored. IMHO nsURI is undesirable since it should prevent the current workspace version being used. However since most users will not be developing a better EMF / UML / OCL a mandatory plugin Java model is not so bad. But painful if a developer tries to fix a user's repro. Since, assuming canonical spelling and ignoring file: access, there are three spellings it seems that a general solution must ensure that all three spellings resolve to the one; project else nsURI else plugin model. This is what StandaloneProjectMap does. > The default practice resulting from "Load > resource..." seems to be > ../../org.eclipse.emf.ecore/model/Ecore.ecore#//EPackage. To what extent is > that a better practice, doesn't it have the same implications? IMHO this is part of the confusion. The target depends on whether the referencer is platform:/plugin or platform:/resource. It should always primarily reference platform/resource with a tooling option to fallback.
(In reply to Ed Willink from comment #8) > Since, assuming canonical spelling and ignoring file: access, there are > three spellings Correction: four. There may be distinct project compiled Java and plugin compiled Java, but the standard classpath ordering should prioritize the project version. The unique/costly part of StandardProjectMap is the creation of a ResourceSet-local EPackageRegistry with descriptors that resolve to a workspace model before falling back to the direct java reference. This requires an eager classpath analysis including a read of all plugin.xml to find and read referenced *.genmodel to populate the local EPackageRegistry before an access occurs. EPackageRegistryBasedURIResourceMap inspires me to do this lazily in get() rather than eagerly on creation. This could ensure that QVTo always uses a developed before installed preference. The only user/design choice is then whether / how to report the schizophrenia resolution. Since this is all very confusing to users and we can make it just work, probably best to report nothing by default.
Pushed a minimally invasive fix: https://git.eclipse.org/c/mmt/org.eclipse.qvto.git/commit/?h=cgerking/570407&id=bbab239e62add801180ca9608109c7179a0a0b3d This is not meant to be a good and complete solution, it just solves the problem reported on the forum. The requested package is still loaded to obtain its namespace URI (then dropped). Is there a way to avoid this? Also no caching is done, which could be added easily.
New Gerrit change created: https://git.eclipse.org/r/c/mmt/org.eclipse.qvto/+/175380
(In reply to Christopher Gerking from comment #10) > The requested package is still loaded to obtain its namespace URI (then > dropped). Is there a way to avoid this? Surely QVTo's metamodel mappings could be used for that. This reminds me of the approach adopted by class UriMappingAwareResourceSet: https://git.eclipse.org/c/mmt/org.eclipse.qvto.git/tree/plugins/org.eclipse.m2m.qvt.oml.runtime/src/org/eclipse/m2m/internal/qvt/oml/runtime/util/UriMappingAwareResourceSet.java
New Gerrit change created: https://git.eclipse.org/r/c/mmt/org.eclipse.qvto/+/175404
(In reply to Christopher Gerking from comment #10) > Also no caching is done, which could be added easily. Done on cgerking/570407. Also added a parser test case. (In reply to Ed Willink from comment #9) > EPackageRegistryBasedURIResourceMap inspires me to do this lazily in get() Done in class PlatformNamespaceUriResourceMap, please review. I am still unsure if this is the most elegant solution.
The test case once moved to 'first' commit fails. Good. But I do't undestand the fix. I don't even understand the *.qvto. Just modeltype bug570407 uses "bug570407.ecore"; ... or modeltype bug570407 uses "platform:/resource/org.eclipse.m2m.tests.qvt.oml/parserTestData/sources/bug570407/bug570407.ecore"; ... claims the metamodel cannot be found in the QVTo editor. ----- bug570407.ecore explicitly uses Ecore.ecore (a workspace variant) so the mapping of Ecore.ecore to the EPackage is the wrong way round. Ecore provides an nsURI to *.genmodel registration but not to *.ecore so yu have to find the plugin.xml on the classpath to find the genmodel regostrations so that you can read the genmodel to find the *.ecore.. What StandaloneProjectMap does. ----- platform:/resource<->plugin mappings are usually done by EcorePlugin.computePlatformURIMap
(In reply to Ed Willink from comment #15) > ... claims the metamodel cannot be found in the QVTo editor. AFAIK QVTo can only resolve nsURIs, that's why it has its properties page for metamodel mappings. To set up these mappings during the test, we have includeMetamodel("bug570407.ecore") in TestQvtParser. > bug570407.ecore explicitly uses Ecore.ecore (a workspace variant) so the > mapping of Ecore.ecore to the EPackage is the wrong way round. The test case assumes that there is no Ecore.ecore in the workspace. The original user problem was based on a relative URI like ../../org.eclipse.emf.ecore/model/Ecore.ecore, which boils down to platform:/resource/org.eclipse.emf.ecore/model/Ecore.ecore when loaded by the QVTo interpreter. For technical reasons, I used the platform:/resource URI directly because QVTo's test infrastructure will copy the files to a different directory, so the ../../ can no longer be resolved correctly. But basically the same fix should apply to the user problem as well. I agree that StandaloneProjectMap and EcorePlugin.computePlatformURIMap probably provide more elegant solutions. To be honest I won't find the time to analyze them before 2021-3, so maybe we should just postpone.
(In reply to Christopher Gerking from comment #16) > so maybe we should just postpone. It will be no worse than it has been for the last 10 years. > EcorePlugin.computePlatformURIMap This should probably be there anyway. It's kind of an EMF standard that EMF expects all applications to clone. > StandaloneProjectMap I've been thinking about an org.eclipse.ocl.emf plugin that factors out the general purpose (Pivot) OCL tools without inflicting Pivot OCL / Xtext dependencies. That might make the existing code more acceptable; QVTo already has an org.eclipse.ocl.ecore dependency. But we need to really understand what we need to fix. I see QVTo as a model (XMI) to model (XMI) tool for which the metamodels are typically *.ecore in the developer's workspace, but which might just happen to be genmodelled giving better performance. Therefore we should probably always map nsURI to *.ecore except when there is no *.ecore or the *.ecore timestamp is earlier than the XXXPackageImpl.java timestamp. (But beware if genmodel rewrites the *.ecore the dates might be out of order, or if checked out from GIT / ...) Aargh! I guess we have to compute a deep semantic hash code so that we can compare the *.java and the *.ecore for semantic equivalence and so allow the compiled identical model to take precedence. Maybe the QVTo builder could cache these hashes wrt timestamps to avoid load-time computations...
(In reply to Ed Willink from comment #15) > platform:/resource<->plugin mappings are usually done by > EcorePlugin.computePlatformURIMap Branch cgerking/570407 now includes a new fix that makes use of EcorePlugin.computePlatformURIMap. As we intend to use a custom URI resource map to fix this bug, the challenge is that the URIConverter of a Resource is only called after the URI resource map has been queried. For this reason, the URI resource map now uses its own intrnal URIConverter, which includes EcorePlugin.computePlatformURIMap. The computation slows down the QVTo compilation significantly, so is there a way to compute less often? (In reply to Ed Willink from comment #15) > bug570407.ecore explicitly uses Ecore.ecore (a workspace variant) so the > mapping of Ecore.ecore to the EPackage is the wrong way round. Also changed the test case to include the wrong platform:/plugin/org.eclipse.emf.ecore/model/Ecore.ecore, which demonstrates the fix. More importantly it fixes the user problem reported on the forum.
Pushed to master for M3.
Addressing a potential regression on cgerking/570407. It is important that the URI normalization only takes place if the requested URI does not exist. Otherwise, platform:/resource/org.eclipse.emf.ecore/model/Ecore.ecore would always be normalized, even if org.eclipse.emf.ecore exists in the workspace and was explicitly addressed by the user. (In reply to Christopher Gerking from comment #18) > The computation slows down the QVTo compilation significantly, so is there > a way to compute less often? The new approach also allows the platform URI map to be computed on demand, which keeps the tests fast.
(In reply to Christopher Gerking from comment #20) > Addressing a potential regression on cgerking/570407. Well that is what often happens when you decide to develop your own solution to a difficult problem rather than re-using a solution that has solved many of them. (In reply to Ed Willink from comment #0) > See StandaloneProjectMap.
I see a Gerrit build for Bug 573572 which appears to be a prototype, but no Gerrit build for this bug, which might be ready to go for RC1 on Tuesday. What's the status?
(In reply to Ed Willink from comment #22) > What's the status? Added more tests on cgerking/570407, all of them pass at least locally. I still couldn't figure out why there are no Gerrit builds. (In reply to Ed Willink from comment #0) > See StandaloneProjectMap. Prototyped on cgerking/570407-StandaloneProjectMap. I agree that it's a much more convenient solution and makes all the tests pass without further ado. But the thing is that it doesn't solve the original user problem reported on the forum. It seems that ProjectMap.getAdapter(...) doesn't create a proper URI-resource-mapping for ../../org.eclipse.ocl.pivot/model/Pivot.ecore, which is referenced like that by the user's metamodel. Instead, the URI is just normalized to platform:/plugin/org.eclipse.ocl.pivot/model/Pivot.ecore, which still leads to metamodel schizophrenia.
(In reply to Christopher Gerking from comment #23) > Added more tests on cgerking/570407, all of them pass at least locally. I > still couldn't figure out why there are no Gerrit builds. Exactly what action did you take to (not) trigger the build? I prefer branch-tests so: https://ci.eclipse.org/qvt-oml/job/qvto-branch-tests/108/ is good.
New Gerrit change created: https://git.eclipse.org/r/c/mmt/org.eclipse.qvto/+/181159
Fixed the standalone tests on cgerking/570407, which obviously require a manual indication as to where platform:/resource/org.eclipse.emf.ecore/model/Ecore.ecore should point. For me this is ready to go for RC1, a better solution based on StandaloneProjectMap should be reconsidered later, though. (In reply to Ed Willink from comment #24) > Exactly what action did you take to (not) trigger the build? I suspect that I did several local commits and then pushed the whole branch. It worked normally right now when I committed/pushed at the same time.
Gerrit change https://git.eclipse.org/r/c/mmt/org.eclipse.qvto/+/181159 was merged to [master]. Commit: http://git.eclipse.org/c/mmt/org.eclipse.qvto.git/commit/?id=749590d2f4b3585b941fabfb48ab1c0ffc2ce075
(In reply to Christopher Gerking from comment #26) > Fixed the standalone tests on cgerking/570407, Pushed to master for RC1. > which obviously require a > manual indication as to where > platform:/resource/org.eclipse.emf.ecore/model/Ecore.ecore should point. Perhaps the Classic OCL workaround avoiding the extra Pivot OCL dependency might help. See org.eclipse.ocl.internal.helper.PluginFinder See org.eclipse.ocl.uml.OCL for its usage to find org.eclipse.uml2.uml.resources
