[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [Newsgroup Home]
[news.eclipse.technology.dltk] Re: Semantic checking - via validators? 
Hi Chuck,

At the moment the the best way is to override parseSourceModule().
When it is called from Indexer - the problemReporter is null, so you can skip additional code.

For examples please look at the RubySourceElementParser and TclSourceElementParser - after standard processing they call TODO task parser, and that is similar to what you need.

Regards,
Alex

Chuck Doucette wrote:
Based on what I found below, here are my conclusions:
We should find the smallest/earliest place that we can change that is called from the editor reconciliation and ideally no place else.
I see two reasonable solutions:

1) Overriding m), i.e. AbstractSourceElementParser.parseSourceModule, since we already have a class which extends this. It would do the following:

public void parseSourceModule(char[] contents, ISourceModuleInfo astCache, char[] filename) {
super.parseSourceModule(contents, astCache, filename);
// do semantic checking
}

The only problem with this approach is that it is also called by SourceIndexer.index.

2) Override VScriptSourceViewerConfiguration.getReconciler, and specifically to add child class of ScriptReconcilingStrategy which implements reconcile. It would consist of the following:

protected void reconcile(final boolean initialReconcile) {
super.reconcile(initialReconcile);
// do semantic checking
}

This would not affect SourceIndexer.index, and it is close to the recommended strategy of implementing my own recognizer, but it builds off of the existing recognizer instead but is *very* dependent on the current implementation so as to change/implement as few lines as possible.

Chuck

"Chuck Doucette" <cdoucette@xxxxxxxxxxx> wrote in message news:g3plql$hkg$1@xxxxxxxxxxxxxxxxxxxx
As I said in another reply, I have created a semantic checker, and I am trying to determine where/how to invoke it.

In addition to invoking it in a builder, we also wanted to invoke it dynamically in the editor, so syntax and semantic errors show up immediately (just like in JDT). Thus, I believe that we should try to follow your advice to invoke it via the reconciler.

In the process of trying to determine exactly where/how to do that, I found out many things that I want to share with other users who may be trying to do the same thing.

Here's what I have found:

a) My (concrete) class VScriptSourceViewerConfiguration extends the DLTK abstract class ScriptSourceViewerConfiguration which defines this method:

/**
* Returns the reconciler ready to be used with the given source viewer.
* This implementation always returns <code>null</code>.
*
* @param sourceViewer the source viewer to be configured by this configuration
* @return a reconciler or <code>null</code> if reconciling should not be supported
*/
public IReconciler getReconciler(ISourceViewer sourceViewer) {

b) The DLTK class ScriptReconciler implements IReconciler. Its super class, MonoReconciler depends on an instance of IReconcilingStrategy to do the reconciliation.

c) The method getReconciler above creates an instance of the DLTK class ScriptCompositeReconcilingStrategy whose superclass, CompositeReconcilingStrategy, implements IReconcilingStrategy. It also depends on an instance of the more straight forward class ScriptReconcilingStrategy.

d) ScriptReconcilingStrategy depends on ISourceModule.reconcile

e) SourceModule extends AbstractSourceModule (both of which implements ISourceModule)
The reconcile method calls ReconcileWorkingCopyOperation.runOperation, which does this:

if (isReadOnly()) {
run(monitor);
} else {
// Use IWorkspace.run(...) to ensure that a build will be done
// in autobuild mode.
// Note that if the tree is locked, this will throw a
// CoreException, but this is ok
// as this operation is modifying the tree (not read-only) and a
// CoreException will be thrown anyway.
IWorkspace wc = ResourcesPlugin.getWorkspace();
wc.run(this, getSchedulingRule(), IWorkspace.AVOID_UPDATE, monitor);

f) IWorkspace.run calls IWorkspaceRunnable.run, and that is defined by ModelOperation.run, which calls executeOperation.

g) ReconcileWorkingCopyOperation.executeOperation calls makeConsistent, which calls SourceModule.makeConsistent:

h) SourceModule.makeConsistent does this:

// Remove AST Cache element
ISourceModuleInfoCache sourceModuleInfoCache = ModelManager
.getModelManager().getSourceModuleInfoCache();
sourceModuleInfoCache.remove(this);
openWhenClosed(createElementInfo(), monitor);

i) ModelElement.openWhenClosed calls Openable.generateInfos:

/**
* Generates the element infos for this element, its ancestors (if they are
* not opened) and its children (if it is an Openable). Puts the newly
* created element info in the given map.
*/
protected abstract void generateInfos(Object info, HashMap newElements,
IProgressMonitor pm) throws ModelException;

j) Openable.generateInfos calls AbstractSourceModule.buildStructure:

/**
* Builds this element's structure and properties in the given info object,
* based on this element's current contents (reuse buffer contents if this
* element has an open buffer, or resource contents if this element does not
* have an open buffer). Children are placed in the given newElements table
* (note, this element has already been placed in the newElements table).
* Returns true if successful, or false if an error is encountered while
* determining the structure of this element.
*/
protected abstract boolean buildStructure(OpenableElementInfo info,
IProgressMonitor pm, Map newElements, IResource underlyingResource)
throws ModelException;

k) AbstractSourceModule.buildStructure calls SourceParserUtil.parseSourceModule

l) SourceParserUtil.parseSourceModule calls AbstractSourceElementParser.parseSourceModule

m) AbstractSourceElementParser.parseSourceModule calls SourceParserUtil.getModuleDeclaration:

n) public static ModuleDeclaration getModuleDeclaration(char[] filename,
char[] content, String nature, IProblemReporter reporter,
ISourceModuleInfo mifo, int flags) {
ISourceParser sourceParser;// = new SourceParser(this.fReporter);
sourceParser = DLTKLanguageManager.getSourceParser(nature);
if (sourceParser instanceof ISourceParserExtension) {
((ISourceParserExtension) sourceParser).setFlags(flags);
}
ModuleDeclaration moduleDeclaration = SourceParserUtil
.getModuleFromCache(mifo, flags);
if (moduleDeclaration == null) {
if (reporter != null) {
reporter.clearMarkers();
}
moduleDeclaration = sourceParser.parse(filename, content, reporter);
SourceParserUtil.putModuleToCache(mifo, moduleDeclaration, flags);
}
return moduleDeclaration;
}

o) My class VScriptSourceParser extends AbstractSourceParser which implements ISourceParser.
The parse method instantiates a vscriptParser and uses it.

NOTES:

o) ISourceParser.parse is also called by:
i) AbstractASTFoldingStructureProvider.getCodeBlocks, which is ultimately called by:AbstractASTFoldingStructureProvider.initialized and AbstractASTFoldingStructureProvider.ElementChangedListener.elementChanged

m) AbstractSourceElementParser.parseSourceModule is also called by:i) SourceIndexer.indexDocument

n/p) SourceParserUtil.getModuleDeclaration (different from the getModuleDeclaration above):
public static ModuleDeclaration getModuleDeclaration(ISourceModule module,
IProblemReporter reporter) {
is called from these places:

i) VScriptCompletionEngine.locateVScriptContext(SourceModule, int)
ii) VScriptSelectionEngine.select(ISourceModule, int, int)
iii) MatchLocatorParser.parse(PossibleMatch)
iv) SemanticPositionUpdater.reconcile(ISourceModule sourceModule, SemanticHighlightingPresenter presenter, Highlighting[] highlightings, List currentPositions)

"Andrei Sobolev" <haiodo@xxxxxxxxx> wrote in message news:g09mhq$9i2$1@xxxxxxxxxxxxxxxxxxxx 
Hi Chuck,

Nice to see, what DLTK help you.

We don't have something special in area of semantic analysis, etc.

Where is some ways to implement such features:
1) Using of ScriptBuilder.
In DLTK we have feature named ScriptBuilder. This is incremental resource builder, used for building Mixin models, etc.
It will give you list of changed resources, and you cold check all this files and set appropriate markers.
All Eclipse markers will be show in DLTK Text editors. ScriptBuilder will contain information about builded resources,
and will be executed on each resource operations, like "resource added", "resource content changed".

2) You could use Validator, Validators framework uses Script Builder for incremental execution of validators. One more
additional point, validators framework manages validator instances and allow unified way to sore configurations, etc.
Also user could turn on/off and execute validators from DLTK UI.

3) If you also plan to make semantic checks for file edited, you need to extend Reconciler from your
ScriptSourceViewerConfiguration class. Reconciler will rebuild model, and do checks, after user will make some changes
in code. By default it is 500 milliseconds delay, so reconciler will be executed only if user not type code 500
milliseconds.


References:
1) Extension point to implement: org.eclipse.dltk.core.builder
2) We have "package require" checker implemented for Tcl. (TclCheckBuilder class).
3) ScriptSourceViewerConfiguration.getReconciler()

Best regards,
Andrei Sobolev.


Thanks to ANTLR and DLTK, I now have an advanced editor which can
immediately flag syntactic problems in our language source files.

My question is if/when/how should I find/flag semantic errors in the source
file?

1. Is there any part of DLTK framework which could/should be used to
invoke/perform semantic checks? I assume the answer to this is no. I know
you support external validators - but we'd like the semantic checking to be
performed implicitly.

2.    When should semantic checks be performed - whenever the parser is
invoked?

3. How should errors be reported - using the parser's error reporter?
Also, how can we determine the line number for populating IProblem
in the file if that information is:
a) not available in DLTKToken
b) not kept in ASTNode

Thanks,
Chuck