Community
Participate
Working Groups
I have implemented a Xtext language that generates Xtend code. So, the intention is that a 2 step generation applies. At first, from the Xtext specified language file to Xtend and then from Xtend to Java. When I start the test workbench (Eclipse Luna) the generation of the Xtend source works well, automatically on the file save event. However, the generation of the Java code from the generated Xtend files does not take place. Only when I open a Xtend file and modify it then the Java generation take place. Also a "Clean" does not trigger regeneration of Java sources from Xtend sources. Attached is a bunch of Eclipse projects which may be used for reproduction. In the test workbench the language example source /de.grammarcraft.flow/src/de/grammarcraft/flow/first.flow may used. A description of the intention of the specified language may be find at http://blog.grammarcraft.de/2014/06/05/extend-your-flow-horizon-flow-design-mit-xtend/ in case of interest (in German). For the initial discussion of this issue, see forum entry https://www.eclipse.org/forums/index.php?t=msg&th=859585&goto=1476152&#msg_1476152
Created attachment 248776 [details] Eclipse projects with an Xtext language for reproducing the issue
It may worth to note, that the generated Xtend sources have active annotations processed by an AA processor. This AA processor is implemented a the Eclipse project "FlowDesignInXtend-ActiveAnnotations". In my test workbench's test project this project is included too in the workspace and the test project refers to it to get the AAs resoved adn incorporated.
The mentioned Eclipse project "FlowDesignInXtend-ActiveAnnotations" is part of the attachment.
Preliminary scheduled for 2.8 - no promises. Why don't you generate straight Java code but Xtend instead?
(In reply to Sebastian Zarnekow from comment #4) > Preliminary scheduled for 2.8 - no promises. > > Why don't you generate straight Java code but Xtend instead? Thanks for scheduling! Do you know a one (or two) click workaround to get the Java sources generated? I just want to avoid to open every generated Xtend source file manually to get it compiled. I do not generate Java code as I want to use Xtend as programming language environment, not Java. :-)
(In reply to Denis Kuniß from comment #5) > Do you know a one (or two) click workaround to get the Java sources > generated? The only workaround that comes to my mind is to generate the Xtend files into an own project. > > I do not generate Java code as I want to use Xtend as programming language > environment, not Java. :-) I don't understand why it's better to produce Xtend given that it compiles to Java in the end but needs an additional step -> longer turnarounds.
(In reply to Sebastian Zarnekow from comment #6) > > > > I do not generate Java code as I want to use Xtend as programming language > > environment, not Java. :-) > > I don't understand why it's better to produce Xtend given that it compiles > to Java in the end but needs an additional step -> longer turnarounds. The language is for specifying Xtend classes which are for integration purposes only. However the functional part of a software system are intended to be implemented at Xtend classes manually. The generated Xtend classes only integrates the hand written ones (and other generated if needed). The concept behind is those of Flow Design invented by Ralf Westphal which allows to apply the concept of Separation of Concerns to integration code and functionality code too. The one (hand written) classes are for the functionality only. The other (integrating) classes are for integration purposes only. This design approach scales for every size of software systems. For more information on Flow Design look here: http://blog.ralfw.de/2013/01/beispielhafte-nichtbeachtung.html (German, "zum schnelleren überfliegen" ;-) Or, here a short overview of the approach in English: http://www.geekswithblogs.net/theArchitectsNapkin/archive/2011/03/19/flow-design-cheat-sheet-ndash-part-i-notation.aspx Or, the long article series where he had embedded the Flow Design approach in a more all-embracing SW design approach: http://www.geekswithblogs.net/theArchitectsNapkin/category/19718.aspx
Hi, I just run into this problem myself this week, here is how I managed to make it work. First, the cause of the problem is that the XtextBuilder will remember the workspace state as it is after the build finishes, including any sources generated by the builder participants. So the IBuildContext#needRebuild (as the javadoc says) is only going to work if another builder has to build the generated artefact (e.g. XtextBuilder generates java from xtend, and then the Java Builder compiles the java source, that works). But it does not work if the XtextBuilder must build the generated sources again. My fix/hack is to remember the workspace tree state right before the builder participant runs, and to then prevent the BuildManager from updating the last built state afterwards. This will cause the xtext builder to rebuild the generated sources in a second build. Subclass the XtextBuilder and put this code where the participant is called: if (participant != null) { try { // remember the current workspace tree as the last built state. final Method method = InternalBuilder.class.getDeclaredMethod("setLastBuiltTree", new Class<?>[] {ElementTree.class}); method.setAccessible(true); final ElementTree lastTree = ((Workspace) ResourcesPlugin.getWorkspace()).getElementTree(); lastTree.immutable(); method.invoke(this, lastTree); } catch (IllegalAccessException e) { LOGGER.error(e); } catch (IllegalArgumentException e) { LOGGER.error(e); } catch (InvocationTargetException e) { LOGGER.error(e); } catch (NoSuchMethodException e) { LOGGER.error(e); } catch (SecurityException e) { LOGGER.error(e); } final BuildContext buildContext = new BuildContext(this, resourceSet, deltas, type); participant.build(buildContext, progress.newChild(1)); if (buildContext.isRebuildRequired()) { rememberLastBuiltState(); // prevents the BuildManager from updating the built state } } else { progress.worked(1); } Regards, Bernhard
(In reply to Bernhard Buss from comment #8) > Hi, > > I just run into this problem myself this week, here is how I managed to make > it work. > > First, the cause of the problem is that the XtextBuilder will remember the > workspace state as it is after the build finishes, including any sources > generated by the builder participants. So the IBuildContext#needRebuild (as > the javadoc says) is only going to work if another builder has to build the > generated artefact (e.g. XtextBuilder generates java from xtend, and then > the Java Builder compiles the java source, that works). But it does not work > if the XtextBuilder must build the generated sources again. > > My fix/hack is to remember the workspace tree state right before the builder > participant runs, and to then prevent the BuildManager from updating the > last built state afterwards. This will cause the xtext builder to rebuild > the generated sources in a second build. > > Subclass the XtextBuilder and put this code where the participant is called: > > if (participant != null) { > try { > // remember the current workspace tree as the last built state. > final Method method = > InternalBuilder.class.getDeclaredMethod("setLastBuiltTree", new Class<?>[] > {ElementTree.class}); > method.setAccessible(true); > final ElementTree lastTree = ((Workspace) > ResourcesPlugin.getWorkspace()).getElementTree(); > lastTree.immutable(); > method.invoke(this, lastTree); > } catch (IllegalAccessException e) { > LOGGER.error(e); > } catch (IllegalArgumentException e) { > LOGGER.error(e); > } catch (InvocationTargetException e) { > LOGGER.error(e); > } catch (NoSuchMethodException e) { > LOGGER.error(e); > } catch (SecurityException e) { > LOGGER.error(e); > } > final BuildContext buildContext = new BuildContext(this, > resourceSet, deltas, type); > participant.build(buildContext, progress.newChild(1)); > if (buildContext.isRebuildRequired()) { > rememberLastBuiltState(); // prevents the BuildManager from > updating the built state > } > } else { > progress.worked(1); > } > > Regards, > Bernhard Note: BuildContext is a custom simple implementation of IBuildContext that allows to retrieve information whether "needRebuild" was called. Just to minimize the impact of this change (we have builder participants that don't want to rebuild afterwards), so no second build is triggered in these cases...
My previous post had a problem with the full build, it only worked for incremental builds. And it was a hack anyways. Here is a better solution: in XtextBuilder#doBuild, replace the participant part with this: if (participant != null) { final BuildContext buildContext = new BuildContext(this, resourceSet, deltas, type); // remember the current workspace tree final ElementTree oldTree = ((Workspace) ResourcesPlugin.getWorkspace()).getElementTree(); oldTree.immutable(); participant.build(buildContext, progress.newChild(1)); if (buildContext.isRebuildRequired() && rebuilds++ <= 2) { final ElementTree newTree = ((Workspace) ResourcesPlugin.getWorkspace()).getElementTree(); newTree.immutable(); final ResourceDelta generatedDelta = ResourceDeltaFactory.computeDelta((Workspace) ResourcesPlugin.getWorkspace(), oldTree, newTree, getProject().getFullPath(), -1); // rebuild the generated delta incrementalBuild(generatedDelta, progress.newChild(1)); } } else { progress.worked(1); } Additionally, add the rebuilds field and reset it in the build() method. I omitted getProject().getWorkspace().checkpoint(false); not sure if it makes a difference :)
I hate not being able to edit comments... am used to it. Here is an addition; Make sure to correctly handle the progress monitor. Initialize with 3 steps (build + participant + rebuild). And the else part should of course then have 2 ticks: } else { progress.worked(2); }
Bernard, thanks for the workaround. I'm not so familiar with Eclipse plugin programming. I'm just using Xtext for that. If it is not too complicated, could you give me a hint how to add your workaround proposal to my Xtext language plugin? But I don't want to waste your time if it is too complicated to be described. I just would wait until it get fixed in Xtext...
(In reply to Denis Kuniß from comment #7) > The language is for specifying Xtend classes which are for integration > purposes only. However the functional part of a software system are intended > to be implemented at Xtend classes manually. The generated Xtend classes > only integrates the hand written ones (and other generated if needed). Again, why do you need to use Xtend for the auto-generated code? Xtend will cleanly link against Java. So you can just generate Java code from your DSL and use Xtend for handwritten parts. That will make your setup easier and builds faster.
(In reply to Stefan Oehme from comment #13) > Again, why do you need to use Xtend for the auto-generated code? > > Xtend will cleanly link against Java. So you can just generate Java code > from your DSL and use Xtend for handwritten parts. That will make your setup > easier and builds faster. For the same reason as Xtend generates Java and not byte code, I guess. But there are several other reasons too. At first, abstraction: By Providing active annotation for hand written classes I have introduced an higher abstraction at the level of language concepts. This abstraction I would like to reuse for generated classes as as well. At second, readability: Generated classes should be readable for users as the hand written ones without changing the level of abstraction and the language. At third, satisfying the principle "Don't repeat yourself": The AA also makes code generation (here in Java). When I generate Java code for the little Xtext integration language I would have to repeat part of the generation code from the AA processor at the Xtext code generator violating this principle. At fourth, evolvability and satisfying the principles of "single responsibility" and "separation of concerns": Every functional change should trigger change at exactly one code container. When I want to change something on the principles how ports (a concept of Flow Design) are implemented (e.g. usage of repeating annotations, introduced with Java 8), I would change it at the AA processor only. There would be no need to apply changes to the Xtext integration language code generator too. At fifth, effort and re-usage: The AA annotations are already done, why not reuse it and save efforts. Currently my system is more a design study. I would like restrict my effort as far as possible. Thanks to the Xtext team: Without your effort and the excellent Xtext system, I would not try this generation approach at all. I'm sure, the principles of Clean Code Development (http://www.clean-code-developer.de) apply to code generation systems too. That's why I would like to make it in that way. And, speed is only one point of all requirements. It has to be measured and prioritized by the stakeholders of a SW system against the other requirements which are on the table (or board ;-). Currently I'm the only stakeholder and speed is not a requirement to me. Requirements should be tackled when they are brought up by the stakeholders of a SW system, I think. When speed becomes an issue mentioned by the users I will consider to generate Java code directly.
I see. The fact that your DSL generates concepts on the same level as your handwritten classes was the missing piece for me =) It makes sense to reuse your AAs then, yes. I thought the DSL just generated more low-level plumbing.
To answer the question why we would need this; In our case we have multiple sources of one (or several) DSLs, that contribute to a common model instance, that also has a DSL. This latter model shall be referenceable by other DSLs, and be contained in one resource. Since there are multiple sources that contribute to one model (and one containment), I decided to use a generation approach to create a new resource and contain the generated model in there. By generating a source that afterwards gets parsed is a close thing at the moment, and also satisfies the requirement of code readability; users want to look at the model in text form. Since the generated model shall be referencable, the generated source needs to be built as well. To answer what is necessary to use my workaround posted above: You can either patch the xtext.jar plugin, or (better) implement your new builder plugin: Create a new plugin that will represent your custom "XtextBuilder" (you can also put it in your language core plugin, but it would be nicer to separate it, who knows what other customizations you might want to do in the future; we did A LOT ;). In that builder plugin: 1. Create a new class extending XtextBuilder -- Put the code I posted above into doBuild 2. Create a simple implementation of IBuildContext, or forego the check of needRebuild in doBuild by assuming this always to be true 3. Create a guice module for your builder, extending AbstractGenericModule -- bind your new builder to IncrementalProjectBuilder 4. Create a plugin.xml -- Add an extension to register your guice module (and thus replacing the xtext builder): <extension point="org.eclipse.xtext.ui.shared.overridingGuiceModule"> <module class="com.avaloq.tools.modelcache.core.context.ContextBuilderModule"> </module> </extension> Let me know if you have any problems. I will monitor this issue, because I am also interested to hear some feedback on the workaround I suggest here. For me it worked well so far, but I am no expert myself in the eclipse builder and workspace area...
I'm sorry, but we won't be able to fix this for 2.8.
