Skip to main content

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [cdt-dev] Clean way to parse a String into a IASTNode

There are thousands of AST nodes, I don't know exactly you what you need. You can choose subset that is mostly interesting, such as
_expression_, Statement, Statement List, Function Definition, Declaration (mostly for class, struct, etc), Declarations List

On Tue, Dec 1, 2015 at 5:04 AM, Krishna Narasimhan <krishna.nm86@xxxxxxxxx> wrote:
Heuristics seem interesting. Actually, since I am parsing an input from some language, I can introduce a field there expecting the user of that language to specify what the code fragment is (whether an Expr, Statement or declaration). So, detecting the ast fragment for what it is, is not the mandatory bit. But I would need a code that can handle all the cases (and be able to give a list of the possible cases to be chosen as one of the inputs).

Although, for this I would need the minimal list of ASTNode types that encompass(superset, interface) all the possible type of nodes.  Is there such a list? 

Do Expressions, Statements and Declarations cover all the cases?



On Tue, Dec 1, 2015 at 2:42 AM, Alena Laskavaia <elaskavaia.cdt@xxxxxxxxx> wrote:
Sorry I was not clear, tests do not do trial and error, they know what they parsing, but if they do know that they are parsing an _expression_ they will add a minimal context around it.
Code for parsing same as your in snippet, you can see it in CodanFastCxxAstTestCase.
Doing trial and error will be expensive, you maybe can use some heuristics like if code does not have ";" or "{" try parse as _expression_, if start with "{" - compound statement, etc
C language also pretty nasty in a way that without context you cannot tell what it is, i.e. "a * b" either _expression_ or declaration of variable b of type a. With gnu extensions you can have
statements inside expressions,etc.

On Mon, Nov 30, 2015 at 3:31 PM, Krishna Narasimhan <krishna.nm86@xxxxxxxxx> wrote:
Thanks a lot. Actually, I was planning to add an exhaustive function with switch cases. Just wanted to know if there is some code piece that I can inspire from or reuse. Are there particular codan checks or refactorings I could look at for this? That you are aware of.



On Mon, Nov 30, 2015 at 9:25 PM, Alena Laskavaia <elaskavaia.cdt@xxxxxxxxx> wrote:
There is no universal way, you have to know what you parsing. If this is _expression_ gramma of _expression_ is different than gramma of function body
than gramma of a c file. If you know what it is you add code around  to make it full c snippet then run parser on it. CDT core and codan tests will do that a lot

On Mon, Nov 30, 2015 at 3:02 PM, Krishna Narasimhan <krishna.nm86@xxxxxxxxx> wrote:
Hello,
   I posted this question and got some input. But that was for a specific case when I know the string is an _expression_. I would like to have a code that can take a String C++ code fragment as input and parse it into an IASTNode. I have a code like this, but this doesnt work if the input is an _expression_. I have to write a special case where I wrap the _expression_ into a function , parse the function and get the child _expression_ node. Is there a clean way that works for all kinds of C++ code fragments. The code below works only if the string is not an _expression_. I have another wrapper function that can work on _expression_ by wrapping the string _expression_ around a function body. I would like to have a universal way for this. 




public static IASTTranslationUnit parse(String code) throws Exception {
final boolean gcc = true;
FileContent codeReader = FileContent.create("<test-code>", code.toCharArray());
ScannerInfo scannerInfo = new ScannerInfo();
ISourceCodeParser parser2 = null;
IScanner scanner = createScanner(codeReader, ParserLanguage.CPP, ParserMode.COMPLETE_PARSE, scannerInfo);
ICPPParserExtensionConfiguration config = null;
if (gcc) {
config = new GPPParserExtensionConfiguration();
} else {
config = new ANSICPPParserExtensionConfiguration();
}
parser2 = new GNUCPPSourceParser(scanner, ParserMode.COMPLETE_PARSE, NULL_LOG, config);
IASTTranslationUnit tu = parser2.parse();
if (parser2.encounteredError())
throw new RuntimeException("PARSE FAILURE");
IASTProblem[] problems = CPPVisitor.getProblems(tu);
if (problems.length > 0) {
throw new RuntimeException("Parse fails: " + Arrays.toString(problems));
}
return tu;
}

public static IScanner createScanner(FileContent codeReader, ParserLanguage lang, ParserMode mode,
IScannerInfo scannerInfo) {
IScannerExtensionConfiguration configuration = null;
if (lang == ParserLanguage.C) {
configuration = GCCScannerExtensionConfiguration.getInstance(scannerInfo);
} else {
configuration = GPPScannerExtensionConfiguration.getInstance(scannerInfo);
}
IScanner scanner;
scanner = new CPreprocessor(codeReader, scannerInfo, lang, NULL_LOG, configuration,
IncludeFileContentProvider.getSavedFilesProvider());
return scanner;
}


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev


_______________________________________________
cdt-dev mailing list
cdt-dev@xxxxxxxxxxx
To change your delivery options, retrieve your password, or unsubscribe from this list, visit
https://dev.eclipse.org/mailman/listinfo/cdt-dev





Back to the top