[xtext-dev] Getting xtext to play along with a custom ANTLR 4 lexer.

[Ralf Gunter Correa Carvalho <ralfgunter@xxxxxxxxx>]

Hello,

A while ago I posted a question on stackoverflow asking how to embed C++ code in a xtext (2.9.1) DSL [1]. Ultimately we went with a custom ANTLR 4 (4.5.1) lexer making heavy use of lexing modes. While the lexer works as intended, it does depend on an ANTLR 4 jar, which breaks generating the xtext grammar artifacts:

0    [main] ERROR mf.mwe2.launch.runtime.Mwe2Launcher  - EOF_TOKEN
java.lang.NoSuchFieldError: EOF_TOKEN
	at org.eclipse.xtext.parser.antlr.Lexer.nextToken(Lexer.java:59)
	at org.antlr.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:143)
	at org.antlr.runtime.BufferedTokenStream.sync(BufferedTokenStream.java:137)
	at org.antlr.runtime.BufferedTokenStream.fill(BufferedTokenStream.java:286)
	at org.antlr.runtime.BufferedTokenStream.toString(BufferedTokenStream.java:251)
	at org.eclipse.xtext.parser.antlr.AbstractInternalAntlrParser.parse(AbstractInternalAntlrParser.java:534)
	at org.eclipse.xtext.parser.antlr.AbstractAntlrParser.doParse(AbstractAntlrParser.java:102)
	at org.eclipse.xtext.parser.antlr.AbstractAntlrParser.parse(AbstractAntlrParser.java:84)
	at org.eclipse.xtext.parser.antlr.AbstractAntlrParser.doParse(AbstractAntlrParser.java:62)
	at org.eclipse.xtext.parser.AbstractParser.parse(AbstractParser.java:33)
	at org.eclipse.xtext.resource.XtextResource.doLoad(XtextResource.java:170)
	at org.eclipse.xtext.linking.lazy.LazyLinkingResource.doLoad(LazyLinkingResource.java:100)
	at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1518)
	at org.eclipse.emf.ecore.resource.impl.ResourceImpl.load(ResourceImpl.java:1297)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoad(ResourceSetImpl.java:259)
	at org.eclipse.emf.ecore.resource.impl.ResourceSetImpl.demandLoadHelper(ResourceSetImpl.java:274)
	at org.eclipse.xtext.resource.XtextResourceSet.getResource(XtextResourceSet.java:265)
	at org.eclipse.xtext.resource.SynchronizedXtextResourceSet.getResource(SynchronizedXtextResourceSet.java:25)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:57)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Runner.run(Mwe2Runner.java:52)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.run(Mwe2Launcher.java:78)
	at org.eclipse.emf.mwe2.launch.runtime.Mwe2Launcher.main(Mwe2Launcher.java:36)

Removing that jar from the plug-in runtime classpath makes generating the grammar possible but then we have to add it back in when launching the plug-in.

I’m not very familiar with java development so I’m at a loss here. Is there a way to make xtext ignore that jar when generating artifacts? What is the proper/idiomatic way of handling this problem?

Thank you!

[1] — http://stackoverflow.com/questions/35806402/how-to-embed-c-code-in-a-xtext-dsl