Bug 481865 - 2.9.0RC1 SemanticSequencer regression
Summary: 2.9.0RC1 SemanticSequencer regression
Status: REOPENED
Alias: None
Product: TMF
Classification: Modeling
Component: Xtext (show other bugs)
Version: 2.8.0   Edit
Hardware: PC Windows NT
: P3 critical (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard: v2.9.2
Keywords:
Depends on:
Blocks: 481867
  Show dependency tree
 
Reported: 2015-11-10 13:17 EST by Ed Willink CLA
Modified: 2016-06-06 07:22 EDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ed Willink CLA 2015-11-10 13:17:13 EST
The generated SemanticSequencer contains:

public void sequence(ISerializationContext context, EObject semanticObject) {
	...
	switch (semanticObject.eClass().getClassifierID()) {
	case BaseCSPackage.IMPORT_CS:
	     sequence_ImportCS(context, (ImportCS) semanticObject); 
	
which gives a syntax error if the extended editor providing sequence_ImportCS is not also regenerated with 2.9.0. With 2.8.4 the generation gave

	protected void sequence_ImportCS(EObject context, ImportCS semanticObject) {
		genericSequencer.createSequence(context, semanticObject);
	}

ISerializationContext  is not compatible with the former EObject. Hence the regression problem.
Comment 1 Ed Willink CLA 2015-11-10 13:21:39 EST
(This is an example of the kind of problem that would go away with flat editors: see Bug 481408)
Comment 2 Moritz Eysholdt CLA 2015-11-10 13:22:10 EST
hi Ed,

if you set

generateSupportForDeprecatedContextObject = true

in your SerializerFragment for your super-language, both 2.8- and 2.9-sublanguages are supported.
Comment 3 Ed Willink CLA 2015-11-10 13:39:00 EST
(In reply to Moritz Eysholdt from comment #2)
> if you set
> 
> generateSupportForDeprecatedContextObject = true

This option is not available for me in org.eclipse.xtext.generator.serializer.SerializerFragment

> in your SerializerFragment for your super-language, both 2.8- and
> 2.9-sublanguages are supported.

But the super-language has already been generated. Compatibility requires an approach to allow the sub-language to comply with the already generated super-language.
Comment 4 Sven Efftinge CLA 2015-11-10 14:02:48 EST
So you regenerated the sublanguage but not the super language? Why is that?
Comment 5 Ed Willink CLA 2015-11-10 14:13:45 EST
I assume by sub we mean derived.

I regenerated the derived language specifically to see whether Xtext 2.9 was compatible.

The converse of regenerating the super-language is much more obviously breaking since new declarations are propgated to clineys.

Regenerating the sub-language seems reasonable since it is solely a consumer.

e.g. Anyone generating a Ybase++ should be able to do so without regenerating Ybase.
Comment 6 Sven Efftinge CLA 2015-11-10 14:53:46 EST
As the mentioned property is set to true for xbase, too, I wonder if it should be true by default. Otherwise all sub-languages will see such compiler-errors and then be forced to set that property. If that is the case the property doesn't make much sense at all. Couldn't we just go back to that context object and live with it?
Comment 7 Ed Willink CLA 2015-11-19 01:52:08 EST
This is still a problem in 2.9.0RC2.
Comment 8 Sven Efftinge CLA 2015-11-19 02:24:51 EST
We need to add that property to the old fragment, too.
Comment 9 Moritz Eysholdt CLA 2015-11-23 08:47:04 EST
fix merged in:
https://github.com/eclipse/xtext/pull/815

The fix allows languages to generate a an API that will also work for sub-languages that are generated with Xtext < 2.9.

Before a sub-language can be generated with Xtext 2.9, the super language must have been generated with 2.9.
Comment 10 Ed Willink CLA 2015-12-17 07:14:37 EST
Doesn't work with Xtext 2.9.0 on an M4 platform.

That is.

I have a multi-layered grammar/editor all generated with Xtext < 2.9.

I regenerate the most derived layer using Xtext 2.9 and still get inconsistent use of ISerializationContext. Comment#6 suggests it should just work.

Reverting to Xtext 2.8.4, (harder than it should be since MWE2-lang 2.9.0M4 cannot be installed with Xtext 2.8.4) and regeneration works correctly.
Comment 11 Ed Willink CLA 2016-01-11 08:00:52 EST
Ping. This was raised to critical by Sebastian. I reopened because it still doesn't work. Any plans?
Comment 12 Sven Efftinge CLA 2016-01-11 08:33:47 EST
Hi Ed, as there seem to be quite some setup involved (super languages generated in <2.9) it would be cool if you could provide a zip and steps to reproduce it. Thank you!
Comment 13 Ed Willink CLA 2016-01-11 08:50:58 EST
(In reply to Sven Efftinge from comment #12)
> it would be cool if you could provide a zip and steps to reproduce it

Adolfo: Does the OOMPH OCL setup (? with an install upgrade to Xtext 2.9) then launch Run->Run Configurations...->MWE2 launch->Generate OCL All Editors reproduce?

Sven: My understanding of your comment #6 was that you recognized that expecting that everyone use an explicit backward compatibility generator option was unhelpful and so you were going to revert the interface change. Simple 'fix'; eliminate ISerializationContext to restore <2.9 compatibility, then look at providing the compatibility that is now required since this problem was allowed into the 2.9 release.
Comment 14 Adolfo Sanchez-Barbudo Herrera CLA 2016-01-11 10:52:36 EST
(In reply to Ed Willink from comment #13)
> (In reply to Sven Efftinge from comment #12)
> > it would be cool if you could provide a zip and steps to reproduce it
> 
> Adolfo: Does the OOMPH OCL setup (? with an install upgrade to Xtext 2.9)
> then launch Run->Run Configurations...->MWE2 launch->Generate OCL All
> Editors reproduce?
> 

No, since all editors are regenerated. However, it can be reproduced if you only regenerate one of the derived ones, e.g.

Run->Run Configurations...->MWE2 launch->Generate OCL CompleteOCL Editor

It can be reproduced with an Oopmh-based OCL Setup. I did the following:
- A Neon Eclipse for committers.
- OCL/Development setup (from Eclipse.org).
- Run->Run Configurations...->MWE2 launch->Generate OCL CompleteOCL Editor
- You then see compilation errors in the SemanticSequencer.

As I've understood from this thread:
- Xtext guys claim to have fixed: when regenerating super-language with Xtext 2.9, sub-languges in Xtext 2.8 should work without regeneration.
- On the contrary, Ed claims to need: when regenerating sub-language with Xtext 2.9, it should always work, even if the super-language is Xtext 2.8 (i.e it's not regenerated with Xtext 2.9).
Comment 15 Moritz Eysholdt CLA 2016-01-14 10:27:16 EST
hi Ed,

(In reply to Ed Willink from comment #10)
> I have a multi-layered grammar/editor all generated with Xtext < 2.9.
> 
> I regenerate the most derived layer using Xtext 2.9 and still get
> inconsistent use of ISerializationContext. Comment#6 suggests it should just
> work.

The following comment explains that this doesn't work:

(In reply to Moritz Eysholdt from comment #9)
> Before a sub-language can be generated with Xtext 2.9, the super language
> must have been generated with 2.9.

the sub-language is the most derived language.

I recommend to generate all super languages with Xtext 2.9.x with this configuration: 

SerializerFragment {
    generateSupportForDeprecatedContextEObject = true
}

After you did this, the sub-languages should work no matter whether they're generated with Xtext 2.8.x or 2.9.x.

(In reply to Ed Willink from comment #13)
> Sven: My understanding of your comment #6 was that you recognized that
> expecting that everyone use an explicit backward compatibility generator
> option was unhelpful and so you were going to revert the interface change.

"everyone" are the people who ship a super language, i.e. a language that is intended to have sub-language in *other* projects. This is true for Xbase and your languages, Ed. It is not true for the very most users because in case they have a super language, they only use it internally and don't encourage other to extend from it. 

Please comment if I'm missing users/projects here.

(In reply to Ed Willink from comment #13)
> Simple 'fix'; eliminate ISerializationContext to restore <2.9 compatibility,
> then look at providing the compatibility that is now required since this
> problem was allowed into the 2.9 release.

ISerializationContext is necessary to support the new grammar features (namely parserrule-parameters and parserrule-fragments) in the serializer. I'm not inclined to remove it.

Am I missing something? What stops you from upgrading your super language to Xtext 2.9.x?
Comment 16 Ed Willink CLA 2016-02-24 05:13:33 EST
(In reply to Moritz Eysholdt from comment #15)
> hi Ed,
 
Hi Moritz; sorry lost your comment.

Using "generateSupportForDeprecatedContextObject = true" and regenerating from the base language up now works, so progress based on base-language first is now possible. e.g. Neon base-editors can be used by Mars sub-editors.

The converse does not work, so clients extending a Mars base-editor must use Mars rather than Neon tooling. Full compatibility requires a "suppressSupportForISerializationContext = true" so that Neon tooling can generate Mars-compatible editors.

The latter is a much less important forward compatibility.

But what does matter is the ability to generate editors that can be installed on a variety of platforms. Currently the Eclipse OCL/QVTd editors work on Xtext 2.3.1 allowing Juno platform users to install Eclipse OCL without needing to upgrade the Xtext that some other editor plugin may require. If Eclipse OCL's editors are generated with Neon Xtext tooling, they will not be installable on Mars.

(There is no Javadoc on generateSupportForDeprecatedContextObject, so no hovertext help appears in the MWE2 editor. I think some documentation would be rather beneficial.)
Comment 17 Sven Efftinge CLA 2016-03-01 06:15:06 EST
Closing this again also see my comment in bug #481867.

(In reply to Ed Willink from comment #16)
> Using "generateSupportForDeprecatedContextObject = true" and regenerating
> from the base language up now works, so progress based on base-language
> first is now possible. e.g. Neon base-editors can be used by Mars
> sub-editors.
> 
> The converse does not work, so clients extending a Mars base-editor must use
> Mars rather than Neon tooling. Full compatibility requires a
> "suppressSupportForISerializationContext = true" so that Neon tooling can
> generate Mars-compatible editors.
> 
> The latter is a much less important forward compatibility.

Ok, Xtext doesn't support the latter kind of compatibility.

> But what does matter is the ability to generate editors that can be
> installed on a variety of platforms. Currently the Eclipse OCL/QVTd editors
> work on Xtext 2.3.1 allowing Juno platform users to install Eclipse OCL
> without needing to upgrade the Xtext that some other editor plugin may
> require. If Eclipse OCL's editors are generated with Neon Xtext tooling,
> they will not be installable on Mars.

As stated in bug #481867. If you want to support Xtext 2.3.1, you need use the generator from 2.3.1.
Comment 18 Ed Willink CLA 2016-06-06 07:22:31 EDT
REOPENED because

a) it was not FIXED, possibly WONTFIXed

(In reply to Sven Efftinge from comment #17)
> Closing this again also see my comment in bug #481867.

b) You made no comment on bug #481867.

c) It is causing trouble to a user on the Xtext newsgroup see "IllegalValueException when extending EssentialOCL".