jdt-core-home/howto/generate parser/generateParser.html
Parent Directory
| 
Revision Log
Revision 1.7 -
(download)
(as text)
(annotate)
Thu Oct 12 19:00:54 2006 UTC (3 years, 1 month ago) by oliviert
Branch: MAIN
Changes since 1.6: +12 -10 lines
Thu Oct 12 19:00:54 2006 UTC (3 years, 1 month ago) by oliviert
Branch: MAIN
Changes since 1.6: +12 -10 lines
HEAD - Fix for 160683
<!doctype html public "-//w3c//dtd html 4.0 transitional//en"> <html> <head> <title>How to: Generate the Parser</title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css"> </head> <body bgcolor="#FFFFFF" text="#000000"> <table border=0 cellspacing=5 cellpadding=2 width="100%" > <tr> <td align=LEFT valign=TOP colspan="2" bgcolor="#0080C0"><b><font color="#FFFFFF"> JDT Core / HowTo: Generate the Parser </font></b></td> </tr> <tr> <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> <td WIDTH="98%"><b>Where to get the parser generator</b><br> <blockquote>The parser files and resources are automatically generated using the LPG parser generator. This tools has been renamed Jikes Parser generator. You can find more information and latest releases at this <A HREF="http://sourceforge.net/project/showfiles.php?group_id=128803&package_id=144579">link</A>. The latest tool is provided in source format. We don't provide any help for compiling these source files. Refer to the link above if you have trouble to get binaries. <br>Our grammar is generated using the version 1.3 of LPG. If newer versions fail to generate resources from our grammar, please send request to the Jikes Parser Generator team. </blockquote> </p> </td> </tr> <tr> <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> <td WIDTH="98%"><b>Where to get the grammar</b><br> <blockquote>The latest grammar is always located in the <code>java_1_5.g</code> file. (N.B.: Older versions of the grammar, such as java_1_4.g are not supported and are no longer functional.) Go to the org.eclipse.jdt.core plugins directory (<font color="#3366FF">eclipse\plugins\org.eclipse.jdt.core</font> where eclipse is the root of your eclipse installation) and open the <font color="#3366FF">grammar</font> directory. Then search for the <font color="#3366FF">java_1_5.g</font> file inside the folder <font color="#3366FF">grammar</font>. Then copy its contents from: <PRE>--main options %options ACTION, AN=JavaAction.java, GP=java, .... $end -- need a carriage return after the $end </PRE> into a file called <font color="#3366FF">java.g</font>. It is important to add a carriage return at the end of the last line. You can save this file where you want, we will assume from thereon you saved it in <code>d:\temp\</code>. <!-- Also make sure that the file has Unix-style end-of-line characters (i.e., not those of MS-DOS), as the parser generator is quite picky about this. --> </blockquote> </p> </td> </tr> <tr> <td ALIGN=RIGHT VALIGN=TOP WIDTH="2%"><img SRC="http://dev.eclipse.org/images/Adarrow.gif" NOSAVE BORDER=0 height=16 width=16></td> <td WIDTH="98%"><b>What to do with these files and update the parser class...</b><br> <blockquote> Assuming the LPG executable (<code>lpg.exe</code> or <code>jikespg.exe</code>) is located in <code>d:\lpg</code>. <ol> <li>First in a console, run: <PRE> d: cd \temp d:\lpg\lpg.exe java.g </PRE> </li> <li>You will get an output that looks like this: <PRE> LPG Parser Generator (V2.30) Tue Apr 02 12:49:13 2002 %OPTIONS ACTION, AN=JavaAction.java, GP=java, %OPTIONS FILE-PREFIX=java, ESCAPE=$, PREFIX=TokenName, OUTPUT-SIZE=125 , %OPTIONS NOGOTO-DEFAULT, SINGLE-PRODUCTIONS, LALR=1 , TABLE=TIME , %OPTIONS ERROR_MAPS %OPTIONS first follow %OPTIONS TRACE=FULL , %OPTIONS VERBOSE Options in effect: ACTION ACTFILE-NAME=JavaAction.java BLOCKB=/. BLOCKE=./ BYTE CONFLIC DEFAULT=5 NODEBUG DEFERRED NOEDIT ERROR-MAPS ESCAPE=$ FILE-PREFIX=java FIRST FOLLOW GENERATE-PARSER=JAVA NOGOTO-DEFAULT HACTFILE-NAME=javahdr.java HBLOCKB=/: HBLOCKE=:/ LALR=1 LIST MAX-DISTANCE=30 MIN-DISTANCE=3 NAMES=OPTIMIZED NONT-CHECK ORMARK=| OUTPUT-SIZE=125 PREFIX=TokenName READ-REDUCE NOSCOPES NOSHIFT-DEFAULT SINGLE-PRODUCTIONS STACK-SIZE=128 STATES SUFFIX= TABLE=TIME TRACE=FU VERBOSE WARNINGS XREF This grammar is LALR(1). Number of Terminals: 105 Number of Nonterminals: 202 Number of Productions: 437 Number of Single Productions: 162 Number of Items: 1265 Number of States: 591 Number of Shift actions: 3482 Number of Goto actions: 4061 Number of Shift/Reduce actions: 369 Number of Goto/Reduce actions: 687 Number of Reduce actions: 7736 Number of Shift-Reduce conflicts: 0 Number of Reduce-Reduce conflicts: 0 Number of Reductions saved by default: 4913 Reallocating storage for TIME table, adding 3603 entries Length of Check table: 16836 Length of Action table: 16608 Number of entries in Action Table: 12013 Percentage of increase: 38.2% Highest symbol in Check Table: 307 Storage Required for Tables: 66888 Bytes, 66K Storage Required for Rules: 1308 Bytes Actions in Compressed Tables: Number of Shifts: 3482 Number of Shift/Reduces: 369 Number of Gotos: 4061 Number of Goto/Reduces: 687 Number of Reduces: 2823 Number of Defaults: 390 Error maps storage: Storage required for ACTION_SYMBOLS_BASE map: 1182 Bytes Storage required for ACTION_SYMBOLS_RANGE map: 1007 Bytes Storage required for NACTION_SYMBOLS_BASE map: 1182 Bytes Storage required for NACTION_SYMBOLS_RANGE map: 630 Bytes Storage required for SYMBOL_INDEX map: 616 Bytes Storage required for STRING_BUFFER map: 4652 Bytes ***Warning: Base Check vector contains value > 127. 16-bit words used. ***Warning: Terminal symbol > 127. 16-bit words used. Escaped symbol $eof is an invalid C variable. Escaped symbol $error is an invalid C variable. </PRE> It can be quite different if the output changed since version 2.30 of LPG. The important part is:<br> <blockquote><b>This grammar is LALR(1).</b></blockquote> This creates some java source files and information files in the current directory. <blockquote><table BORDER=1 CELLSPACING=2 CELLPADDING=10> <tr> <th VALIGN=top align=left>java.l</th> <td VALIGN=top>Log information generated by lpg/jikespg. Enumarates all the states created for the automaton, etc.</td> </tr> <tr> <th VALIGN=top align=left>JavaAction.java</th> <td>Contains the method consumeRule(int) of class org.eclipse.jdt.internal.compiler.parser.Parser, which handles all semantic action dispatches.</td> </tr> <tr> <th VALIGN=top align=left>javahdr.java</th> <td>Used to generate the resources files.</td> </tr> <tr> <th VALIGN=top align=left>javadcl.java</th> <td>Used to generate the resources files.</td> </tr> <tr> <th VALIGN=top align=left>javasym.java</th> <td>This is the contents of the class org.eclipse.jdt.core.compiler.TerminalTokens.</td> </tr> <tr> <th VALIGN=top align=left>javadef.java</th> <td>This is the contents of the class org.eclipse.jdt.internal.compiler.parser.ParserBasicInformation.</td> </tr> <tr> <th VALIGN=top align=left>javaprs.java</th> <td>You don't need this file. Its contents is already inlined in the Parser class.</td> </tr> </table></blockquote> </li> <li><blockquote>Now we need to update the different classes and resource files. </blockquote> <ol> <li>Copy the contents of the <font color="#3366FF">JavaAction.java</font> file into the <font color="#3366FF">consumeRule(int)</font> method of the org.eclipse.jdt.internal.compiler.parser.Parser class. </li> <li>The class <font color="#3366FF">org.eclipse.jdt.internal.compiler.parser.ParserBasicInformation</font> needs to be updated with the contents of the file <font color="#3366FF">javadef.java</font>. Don't copy the interface name. Simply copy the field declarations. The actual source of this class will guide you.</li> <li>Similarly, update the class <font color="#3366FF">org.eclipse.jdt.internal.compiler.parser.TerminalTokens</font> with the contents of the file <font color="#3366FF">javasym.java</font>. <li>The last step is to update the resource files:<br> Copy the jdtcore.jar file in d:\temp. Compile this <A HREF="UpdateParserFiles.java">source</A> inside d:\temp. You will have a file UpdateParserFiles.class. Then run the following command-line: <PRE> D:\temp>java -classpath jdtcore.jar:. UpdateParserFiles javadcl.java javahdr.java </PRE> Once this done, you will end up with 25 new files inside d:\temp. They are called parser<n>.rsc, with n equals to 1..24 and readableNames.properties. All these files need to be moved to the org\eclipse\jdt\internal\compiler\parser folder. Now you are ready to execute and test the new parser. </li> </ol> </li> </ol> <P> <b>NOTE:</b> <blockquote>Changing the parser is a risky operation if you miss one of the steps above. The resulting parser can be completely unpredictable. It can go from crashing to reporting invalid errors. Be sure that you followed all the steps and that all the files are updated and recompiled before you run it. </blockquote> </P> </blockquote> </td> </tr> </table> </body> </html>
| help@eclipse.org | ViewVC Help |
| Powered by ViewVC 1.0.3 |