Bug 496209 - Separate syntax coloring for object-style and function-style macros
Summary: Separate syntax coloring for object-style and function-style macros
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: cdt-editor (show other bugs)
Version: 9.1.0   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-06-15 14:23 EDT by Ladar Levison CLA
Modified: 2020-09-04 15:21 EDT (History)
3 users (show)

See Also:


Attachments
Shows the syntax coloring with macro references. (7.01 MB, video/ogg)
2016-06-15 14:27 EDT, Ladar Levison CLA
no flags Details
The line number bug. (5.75 MB, video/ogg)
2016-06-15 14:27 EDT, Ladar Levison CLA
no flags Details
Code folding when launched bug. (3.09 MB, video/ogg)
2016-06-15 14:29 EDT, Ladar Levison CLA
no flags Details
Shows the print margin bug. (7.20 MB, video/ogg)
2016-06-15 14:35 EDT, Ladar Levison CLA
no flags Details
Screenshot of code folding settings. (38.12 KB, image/png)
2016-11-01 23:34 EDT, Ladar Levison CLA
no flags Details
Screenshot of provide_check.c file and its broken code folds. (233.63 KB, image/png)
2016-11-01 23:35 EDT, Ladar Levison CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ladar Levison CLA 2016-06-15 14:23:17 EDT
Running Neon, I've run across a few annoying bugs with the syntax highlighter, and the editor. Sending video captures.

The print margin line is governed by the line wrap setting in the code formatter, and not the print margin setting under general settings. I generally prefer to have the formatter use a very high number, like 200, and then manually pick the wrap point if it goes significantly past the margin. Otherwise conditionals get wrapped in ways that make them hard to read/understand. See the print.margin.bug.ogv video.

When launching Neon, if a C file is open, the editor tries to fold the code, but ends up munged into an unusuable state. Any non-active tabs with files open work correctly. Closing and opening the file fixes the problem. See the folding.when.launched.bug.ogv video.

The syntax colorer doesn't differentiate between a defined constant, like NULL, and a macro, like strdupa(). See the macro.highlighting.bug.ogv video.

String parameters passed to macros get colored based on the macro reference rule, and not the string constant rule. Disabling the macro reference rule disables highlighting for macro references, but causes constants to get colored properly. See the macro.highlighting.bug.ogv video.

Running Eclipse Neon RC3 atop the IBM JRE 1.8.0 SR3. (No idea if that's related, just an FYI.)
Comment 1 Ladar Levison CLA 2016-06-15 14:27:10 EDT
Created attachment 262469 [details]
Shows the syntax coloring with macro references.
Comment 2 Ladar Levison CLA 2016-06-15 14:27:57 EDT
Created attachment 262470 [details]
The line number bug.
Comment 3 Ladar Levison CLA 2016-06-15 14:29:13 EDT
Created attachment 262471 [details]
Code folding when launched bug.
Comment 4 Ladar Levison CLA 2016-06-15 14:35:54 EDT
Created attachment 262472 [details]
Shows the print margin bug.
Comment 5 Nathan Ridge CLA 2016-06-15 17:00:22 EDT
(In reply to Ladar Levison from comment #0)
> String parameters passed to macros get colored based on the macro reference
> rule, and not the string constant rule. Disabling the macro reference rule
> disables highlighting for macro references, but causes constants to get
> colored properly. See the macro.highlighting.bug.ogv video.

This is bug 490415.

> The syntax colorer doesn't differentiate between a defined constant, like
> NULL, and a macro, like strdupa(). See the macro.highlighting.bug.ogv video.

Is this meant to be a different issue from the above? That is, are you expecting the "strdupa" token itself to be colored differently from the "NULL" token?

> The print margin line is governed by the line wrap setting in the code
> formatter, and not the print margin setting under general settings. I
> generally prefer to have the formatter use a very high number, like 200, and
> then manually pick the wrap point if it goes significantly past the margin.
> Otherwise conditionals get wrapped in ways that make them hard to
> read/understand. See the print.margin.bug.ogv video.

This seems to be bug 481401. Note that, based on the discussion in that bug, it may be by design.

The line number and code folding issues remain to be investigated.
Comment 6 Ladar Levison CLA 2016-06-16 15:06:06 EDT
> 
> > The syntax colorer doesn't differentiate between a defined constant, like
> > NULL, and a macro, like strdupa(). See the macro.highlighting.bug.ogv video.
> 
> Is this meant to be a different issue from the above? That is, are you
> expecting the "strdupa" token itself to be colored differently from the
> "NULL" token?
> 

To answer your question: yes strdupa is a macro, while NULL is a constant. I've added the glibc define for strdupa, and NULL to illustrate. I understand it may be tricky to differentiate between the two. 

My goal is to have macros be highlighted the same as functions, but currently that means having constants will have the same highlighting applied. 

#define NULL 0

# define strdupa(s)							      \
  (__extension__							      \
    ({									      \
      __const char *__old = (s);					      \
      size_t __len = strlen (__old) + 1;				      \
      char *__new = (char *) __builtin_alloca (__len);			      \
      (char *) memcpy (__new, __old, __len);				      \
    }))
Comment 7 Nathan Ridge CLA 2016-06-16 18:42:04 EDT
(In reply to Ladar Levison from comment #6)
> To answer your question: yes strdupa is a macro, while NULL is a constant.
> I've added the glibc define for strdupa, and NULL to illustrate. I
> understand it may be tricky to differentiate between the two. 

Out of the highlightings listed in Preferences -> C/C++ -> Editor -> Syntax Coloring, which would you expect to apply to "NULL", and which to "strdupa"?

Or are you asking for a new highlighting to be added?
Comment 8 Ladar Levison CLA 2016-06-17 03:25:12 EDT
(In reply to Nathan Ridge from comment #7)
> (In reply to Ladar Levison from comment #6)
> Out of the highlightings listed in Preferences -> C/C++ -> Editor -> Syntax
> Coloring, which would you expect to apply to "NULL", and which to "strdupa"?
> 
> Or are you asking for a new highlighting to be added?

The cleanest might be a new highlighter "Macro constants" or "Constant definition" and "Constant reference" which would apply to the decleration and referencing of NULL, while strdupa would remain a Macro definition/reference. 

You might also want to consider moving all three macro/constant rules to the preprocessor section. It feels like the code section is a little over crowded, but that's just my opinion.

If you don't want to create a new highlighter, defines like NULL could also be considered global variables.
Comment 9 Nathan Ridge CLA 2016-06-17 05:16:52 EDT
(In reply to Ladar Levison from comment #8)
> (In reply to Nathan Ridge from comment #7)
> > (In reply to Ladar Levison from comment #6)
> > Out of the highlightings listed in Preferences -> C/C++ -> Editor -> Syntax
> > Coloring, which would you expect to apply to "NULL", and which to "strdupa"?
> > 
> > Or are you asking for a new highlighting to be added?
> 
> The cleanest might be a new highlighter "Macro constants" or "Constant
> definition" and "Constant reference" which would apply to the decleration
> and referencing of NULL, while strdupa would remain a Macro
> definition/reference. 

Ok, thanks. That's a reasonable request.

The reason I was initially confused is that all the other issues reported in this bug were regressions from a previous version of CDT, while references to object-style and function-style macros have (to my knowledge) always been colored the same.
Comment 10 Ladar Levison CLA 2016-06-17 08:14:58 EDT
(In reply to Nathan Ridge from comment #9)
> Ok, thanks. That's a reasonable request.
> 
> The reason I was initially confused is that all the other issues reported in
> this bug were regressions from a previous version of CDT, while references
> to object-style and function-style macros have (to my knowledge) always been
> colored the same.


Sorry about that. I lumped them together. I wasn't sure if this was a regression or not. I'm returning to work on a project after a couple of years. Its possible that some of the symbols I'm using were converted to macros by another developer on the project. That might explain why I remember them being highlighted the same as functions. 

If you'd like I do have a feature request - it sure would be nice if I could filter the project view for resources modified since the last commit! :) I left that one out of the bug report, and never opened up the feature request because I didn't have time to search for a duplicate.
Comment 11 Nathan Ridge CLA 2016-06-17 10:05:20 EDT
(In reply to Ladar Levison from comment #10)
> If you'd like I do have a feature request - it sure would be nice if I could
> filter the project view for resources modified since the last commit! :) I
> left that one out of the bug report, and never opened up the feature request
> because I didn't have time to search for a duplicate.

That's a feature request for whatever plugin you're using to provide version control integration (e.g. Subversive, MercurialEclipse, EGit), not for CDT itself.
Comment 12 Nathan Ridge CLA 2016-07-13 01:01:32 EDT
(In reply to Nathan Ridge from comment #5)
> The line number and code folding issues remain to be investigated.

I'm not able to reproduce either of these issues. I tried following what you did in your videos as closely as possible, but perhaps there is some relevant configuration difference that I am missing.

For the code folding issue, it might help to know what you have checked under Preferences -> C/C++ -> Editor -> Folding.
Comment 13 Nathan Ridge CLA 2016-07-19 03:13:34 EDT
I'm going to use this bug for the syntax coloring feature described in comment 8 and comment 9.

For the other issues (code folding and line numbers), if you still experience them, please file new bugs, preferably with the additional information requested in comment 12. (It's better to have separate bugs for separate issues anyways.)
Comment 14 Ladar Levison CLA 2016-11-01 23:33:44 EDT
Sorry about the lag. I was working on other things for awhile, and forgot to check this ticket. I kept hoping R1 would bring fixes. I guess now I'm hoping for R2. Unfortunately things seem to be getting worse on my end. I'm not sure why...

To answer your question, I will add a screenshot of the code folding setup.

What might be worth trying, is importing the project directly off github. The CDT project files are part of the repo, so it should arrive with the project specific settings. Its possible something about the size, or complexity of the project is triggering this issue:

https://github.com/lavabit/magma

You'll want to import the magma project, and magma.so. You'll also need to run make (or the build.lib.sh) script at least once to expand all the bundled dependencies, and truly stress the parser.

Its been awhile, but think I was able to replicate the issue with a clean Eclipse setup, but its been awhile. I've been thinking about giving that a try at some point. 

Either way, I've managed to survive by using the keyboard shortcut (Cntrl+Numpad-Multiply) to enable/disable folding when the editor goes haywire. That worked for awhile, but it seems the issue is getting worse. Lately, even after I reset the folding, I'll start exploring and the folds will start going bonkers. Particularly in the check/magma directory, where the unit test macros cause problems. Look at provider_check.c, or see the attached screenshot. The suite_check_provide() and check_spf_s() seem to consistently be broken.
Comment 15 Ladar Levison CLA 2016-11-01 23:34:18 EDT
Created attachment 265146 [details]
Screenshot of code folding settings.
Comment 16 Ladar Levison CLA 2016-11-01 23:35:03 EDT
Created attachment 265147 [details]
Screenshot of provide_check.c file and its broken code folds.
Comment 17 Ladar Levison CLA 2016-11-02 23:35:01 EDT
I spent a 3+ hours troubleshooting this issue tonight, hoping to figure out a culprit so I could try to avoid it. Nothing I changed seemed to work.

To start with, I tried creating a new workspace, then cloning the magma project, and removing the .project/.cproject/.settings files/directories from the cloned version. I recreated the project configurations inside the new workspace and the problem popped right back up.

If you want to do the same, clone the project and run the following to extract/compile the bundled dependencies for the magma.so project. This is the short version, as running make first will also run the bundled dependency checks. Execute the following (and then go for a cup of coffee):

dev/scripts/builders/build.lib.sh extract && dev/scripts/builders/build.lib.sh prep && dev/scripts/builders/build.lib.sh build && dev/scripts/builders/build.lib.sh combine

Next create a C/C++ Makefile project for the "lib" folder, and configure it to index the sources subdir. Setup local/include as part of the include path, and export the include folder. Change the builder command from "make" to "ls ../magmad.so" so when you try building it you you get an error.

Finally create a C/C++ project for the magma folder itself, and configure it to reference the magma.so project. To get the indexer working properly setup "src" and "check/magma" as include folders (plus the folder imported via the magma.so reference). Set the build command to "make VERBOSE=yes" so the CDT parser can pull from the gcc output, and add magma.h and magma_check.h as include files. You'll also want to define MAGMA_PEDANTIC. With folding enabled by default you should start seeing problems all over the place. 

I focused on the provide_check.c file, as it seems to get folded and parsed incorrectly on a consistent basis. 

I was able to consistently recreate the issue by disabling folding (Cntrl+Numpad Divide), then enabling folding  (Cntrl+Numpad Divide), and then running make from within Eclipse (Cntrl+B). The check_spf_s would fail everytime the editor refreshed, following a build, just like it shows in the screenshot I uploaded yesterday.

I tried changing everything I could think of to isolate the problem. For starters the parser struggled with anonymous enums/unions so I eliminated the anonymous stringer struct by changing them to defines. I eliminated the anonymous union inside the ip_t struct. I changed STACK to STACKED to avoid confusion with external include files. I tried commenting out the variables, removing the START_TEST macro, and everything failed. I even looked at the raw file bytes to make sure there weren't any hidden chars I couldn't see. Nothing seemed to fix the issue. The "if" statements which called spf_check() failed to get parsed properly every time. For whatever reason, the parser didn't consider them to be part of the check_spf_s function. 

Any ideas what could be causing this?

I'm using Neon (4.6.1) with the CDT tools updated to 9.1.0. I'll update the ticket accordingly.
Comment 18 Nathan Ridge CLA 2016-11-03 00:13:24 EDT
Ladar, could you kindly file a new bug for the issue discussed from comment 14 onwards? It's easier for everyone to follow if there is one issue per bug, and, as indicated in comment 13 and the bug title, I'd like to keep this bug focused on the feature request to have separate syntax colorings for object-style and function-style macros.

Thanks!
Comment 19 Ladar Levison CLA 2016-11-07 05:20:52 EST
I created a separate bug for the folding (bug 507138) and line number *bug 507132) issues, with abbreviated descriptions of the issue. (I didn't want to copy/paste all the details here.)

Briefly, it seems the line numbers is related to the plugin Ooomph Setup UI bundled with CDT EPP. 

And while I haven't figured out why the parser fails on the first editor every launch, I did seem to narrow the recurring fold failures to the if/else do/while, for, switch folding option.
Comment 20 Ladar Levison CLA 2016-11-07 05:21:26 EST
I created a separate bug for the folding (bug 507138) and line number (bug 507132) issues, with abbreviated descriptions of the issue. (I didn't want to copy/paste all the details here.)

Briefly, it seems the line numbers is related to the plugin Ooomph Setup UI bundled with CDT EPP. 

And while I haven't figured out why the parser fails on the first editor every launch, I did seem to narrow the recurring fold failures to the if/else do/while, for, switch folding option.