Bug 22979

Summary: using IScanner inside comments
Product: [Eclipse Project] JDT Reporter: Adam Kiezun <akiezun>
Component: CoreAssignee: Philipe Mulet <philippe_mulet>
Status: RESOLVED WONTFIX QA Contact:
Severity: enhancement    
Priority: P3 CC: jmm
Version: 2.0   
Target Milestone: 2.1 M2   
Hardware: PC   
OS: Windows 2000   
Whiteboard:
Bug Depends on:    
Bug Blocks: 58205    

Description Adam Kiezun CLA 2002-08-29 10:13:42 EDT 
i'd like to be able to feed IScanner with a content of a comment and get access 
to tokens
Comment 1 Philipe Mulet CLA 2002-09-03 04:29:07 EDT 
The scanner is a Java one, and recognize comments as tokens.
Now, you may want to position the scanner behind the comment starting 
delimiter, and tokenize from there. However, the comment content is not meant 
to follow Java specs, and thus illegal tokens are likely to be found in there.

You probably should use a StringTokenizer for this matter.
Comment 2 Adam Kiezun CLA 2002-09-03 04:36:09 EDT 
will that not cause some encoding problems? i mean, IScanner handles all the 
unicode problems. will i have to do it myself when using StringTokenizer?

maybe a separate scanner? IJavaDocScanner :)?
Comment 3 Philipe Mulet CLA 2002-09-23 04:43:22 EDT 
Closing, IScanner isn't meant for parsing comments (unless you know what you're 
doing). Parsing javadocs isn't supported, we have other requests on this front, 
unclear which action we will take.
Comment 4 Philipe Mulet CLA 2003-08-26 08:56:46 EDT 
*** Bug 41190 has been marked as a duplicate of this bug. ***