[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index] [List Home]
Re: [imp-dev] Case-insensitiv lexer - lpg

I think you may be using a somewhat older set of LPG templates; I was quoting from the most recent version of LexerBasicMapF.gi in the LPG distribution.

On Mar 29, 2011, at 10:38 AM, Beverly Sanders wrote:

> Our language is case insensitive.  We use a file called
> LexerFoldedCaseMap.gi with entries like
> 
>            tokenKind['A'] = $prefix$a$suffix$;
>            tokenKind['B'] = $prefix$b$suffix$;
>            tokenKind['C'] = $prefix$c$suffix$;
> 
> This works for us.  I think I found LexerFoldedCaseMap.gi in the imp/lpg
> distribution somewhere, but if you want me to send you a copy, just ask.
> (I have no idea what the $sym_type. in Bob's suggestion is.)
> Beverly
> 
> On Tue, March 29, 2011 10:07 am, Robert M. Fuhrer wrote:
>> Hi Ovidiu,
>> 
>> I'm cc'ing Philippe Charles, the main author of LPG, in case he has
>> something to add.
>> 
>> I think you need to use a modified copy of the LexerBasicMapF.gi include
>> file to effect the case-folding you're looking for.
>> 
>> Specifically, to fold everything to lower-case, find the letter-related
>> entries in the tokenKind[] array that look like:
>> 
>>            $sym_type.$prefix$A$suffix$,               // 065    0x41
>>            $sym_type.$prefix$B$suffix$,               // 066    0x42
>>            $sym_type.$prefix$C$suffix$,               // 067    0x43
>>            ...
>> 
>> and replace them with their lower-case equivalents:
>> 
>>            $sym_type.$prefix$a$suffix$,               // 097    0x61
>>            $sym_type.$prefix$b$suffix$,               // 098    0x62
>>            $sym_type.$prefix$c$suffix$,               // 099    0x63
>>            ...
>> 
>> I haven't actually tried this myself, but I believe it should work.
>> 
>> On Mar 29, 2011, at 9:26 AM, Ovidiu Diac wrote:
>> 
>>> Hi,
>>> 
>>> I'm currently working on an ide for a Java-like scripting language. The
>>> problem i have is that this script is supposed to be case-insensitive.
>>> I managed to make the keywords case-insensitive by using the
>>> KWLexerFoldedCaseMapF.gi include in KWLexer but i don't know what to do
>>> about identifiers. - i'm using the lpg parser
>>> Any help is much appreciated.
>>> Thanks,
>>> Ovidiu Diac
>>> 
>>> _______________________________________________
>>> imp-dev mailing list
>>> imp-dev@xxxxxxxxxxx
>>> https://dev.eclipse.org/mailman/listinfo/imp-dev
>> 
>> --
>> Cheers,
>>  -- Bob
>> --------------------------------
>> Robert M. Fuhrer
>> Research Staff Member
>> Programming Technologies Dept.
>> IBM T.J. Watson Research Center
>> 
>> IMP Team Lead (http://www.eclipse.org/imp)
>> X10: Productivity for High-Performance Parallel Programming
>> (http://x10-lang.org)
>> 
>> _______________________________________________
>> imp-dev mailing list
>> imp-dev@xxxxxxxxxxx
>> https://dev.eclipse.org/mailman/listinfo/imp-dev
>> 
> 
> 
> -- 
> Beverly A. Sanders, Ph.D.
> Associate Professor
> Department of Computer & Information Science & Engineering
> P.O. Box 116120
> University of Florida
> Gainesville, FL 32611-6120
> www.cise.ufl.edu/~sanders
> tel: (352) 450-9213
> fax: (352) 392-1220
> 
> _______________________________________________
> imp-dev mailing list
> imp-dev@xxxxxxxxxxx
> https://dev.eclipse.org/mailman/listinfo/imp-dev
> 

Cheers,
 - Bob