Bug 156008 - build output parser can't handle if...then constructs generated by automake 1.9
Summary: build output parser can't handle if...then constructs generated by automake 1.9
Status: RESOLVED FIXED
Alias: None
Product: CDT
Classification: Tools
Component: cdt-core (show other bugs)
Version: 3.1   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 4.0   Edit
Assignee: Doug Schaefer CLA
QA Contact:
URL:
Whiteboard:
Keywords: contributed
Depends on:
Blocks:
 
Reported: 2006-09-01 16:08 EDT by Tim Froidcoeur CLA
Modified: 2008-06-19 23:54 EDT (History)
5 users (show)

See Also:


Attachments
Patch for GCCPerFileBOPConsoleParser.java (8.38 KB, patch)
2007-02-04 18:36 EST, Gerhard Schaber CLA
no flags Details | Diff
Patch for GCCPerFileBOPConsoleParser.java (10.09 KB, patch)
2007-02-04 18:50 EST, Gerhard Schaber CLA
no flags Details | Diff
Patch for more flexible handling of compiler invocation (13.06 KB, patch)
2007-02-23 15:08 EST, Gerhard Schaber CLA
no flags Details | Diff
Patch for more flexible handling of compiler invocation (CDT HEAD) (13.08 KB, patch)
2007-02-27 06:16 EST, Gerhard Schaber CLA
no flags Details | Diff
More flexible discovery patch that can handle "if", ";". wrappers, quotes, blanks (11.53 KB, patch)
2007-04-19 14:52 EDT, Gerhard Schaber CLA
no flags Details | Diff
More flexible discovery patch that can handle "if", ";". wrappers, quotes, blanks (20.37 KB, patch)
2007-04-20 08:59 EDT, Gerhard Schaber CLA
no flags Details | Diff
More flexible discovery patch that can handle "if", ";". wrappers, quotes, blanks (21.25 KB, patch)
2007-05-07 17:12 EDT, Gerhard Schaber CLA
no flags Details | Diff
Enhanced patch with relative include paths (30.63 KB, patch)
2007-05-16 19:17 EDT, Gerhard Schaber CLA
no flags Details | Diff
Enhanced patch with relative include paths (30.75 KB, patch)
2007-05-18 08:55 EDT, Gerhard Schaber CLA
bjorn.freeman-benson: iplog+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tim Froidcoeur CLA 2006-09-01 16:08:50 EDT
I had trouble with content assist not working for a standard Makefile project with autotools makefiles.
Apparantly the automake 1.9 generates Makefiles that come up with following type of build output:

<snippet>
if g++ -DHAVE_CONFIG_H -I. -I. -I..    -I/usr/include/gtkmm-2.4 -I/usr/lib/gtkmm-2.4/include -I/usr/include/glibmm-2.4 -I/usr/lib/glibmm-2.4/include -I/usr/include/gdkmm-2.4 -I/usr/lib/gdkmm-2.4/include -I/usr/include/pangomm-1.4 -I/usr/include/atkmm-1.6 -I/usr/include/gtk-2.0 -I/usr/include/sigc++-2.0 -I/usr/lib/sigc++-2.0/include -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/lib/gtk-2.0/include -I/usr/include/pango-1.0 -I/usr/include/cairo -I/usr/include/atk-1.0    -g -O2 -MT pictureTag.o -MD -MP -MF ".deps/pictureTag.Tpo" -c -o pictureTag.o pictureTag.cpp; \
    then mv -f ".deps/pictureTag.Tpo" ".deps/pictureTag.Po"; else rm -f ".deps/pictureTag.Tpo"; exit 1; fi
</snippet>

notice the if...then construct

The code assist features would not work for most of those external libraries and the discovery paths didn't show any of those on the path. After switching to automake-1.4 the conditional stuff disappeared  (no more if....then, just plain compile statements) from the build output and suddenly the discovery and code assist worked 

workaround: automake 1.4

this happens both in the per file and per project scanner
Comment 1 Doug Schaefer CLA 2006-09-01 16:24:37 EDT
Sorry, you have me very confused. You talk about build output parsing and content assist in this bug report, which have absolutely nothing to do with each other. Maybe you are seeing two separate problems?
Comment 2 Tim Froidcoeur CLA 2006-09-01 18:21:14 EDT
Most likely I'm the one being confused.

The problem is that code assist doesn't work. 
I think the reason is that the include paths don't get discovered properly due to those if...then constructs in the makefiles. 
Is include path discovery not part of the build output parser functionality?
Comment 3 Gerhard Schaber CLA 2007-02-04 18:36:09 EST
Created attachment 58217 [details]
Patch for GCCPerFileBOPConsoleParser.java

A proposal for GCCPerFileBOPConsoleParser.java.
It supports build commands where the compiler is started by a wrapper such as a shell or another executable.
For instance, for a build command such as 
sh -c 'gcc -g -O0 -I"includemath" -I "include abc" -Iincludeprint -c impl/testmath.c'

Mozilla, for instance, often uses nsinstall.exe to start a compiler command.
Comment 4 Gerhard Schaber CLA 2007-02-04 18:50:36 EST
Created attachment 58219 [details]
Patch for GCCPerFileBOPConsoleParser.java

The same proposal. I just added a missing patch for another file.
Comment 5 Gerhard Schaber CLA 2007-02-23 15:08:39 EST
Created attachment 59706 [details]
Patch for more flexible handling of compiler invocation

The patch is a proposal where a compiler invocation command is analyzed more flexible. It allows invocation of the compiler via script, and allows quoted commands, and blanks in paths. The file collector is also patched in order to allow an immediate resolution and propagation of gathered paths--this allows to skip the separate ScannerConfigBuilder run.
Comment 6 Gerhard Schaber CLA 2007-02-27 06:16:29 EST
Created attachment 59857 [details]
Patch for more flexible handling of compiler invocation (CDT HEAD)
Comment 7 Gerhard Schaber CLA 2007-03-21 05:45:57 EDT
Mikhail,

could you apply this patch or have a look at it? I know there are a lot of other issues you are working on.

Gerhard
Comment 8 Mikhail Sennikovsky CLA 2007-03-21 06:12:35 EDT
Hi Gerhard,

I can look at your patch once I have time :), but I don't think I could make it before the M6 :(  since I have several urgent issues/enhancements needed to be fixed/done for the M6

Mikhail
Comment 9 Doug Schaefer CLA 2007-04-12 15:36:15 EDT
Gerhard Schaber writes:
Hi Doug!
 
You checked in a patch for org.eclipse.cdt.make.core/src/org/eclipse/cdt/make/internal/core/scannerconfig/gnu/GCCPerFileBOPConsoleParser.java.
 
I am not sure if this fixes also cases where more than one wrapper is used, or where the compiler command itself is quoted, or where white space is used in include paths.
 
I already have been waiting for a while for someone having a look at my patch for the corresponding bug 156008. Since you made a patch for this issue, you may be willing to have a look at my patch. You find the patch attached to bug 156008.
Comment 10 Doug Schaefer CLA 2007-04-12 15:39:21 EDT
Actually, I was addressing a specific issue I was having with building Firefox under cygwin and I was too lazy/busy to find any related bug reports. Thanks for pointing me here.

From my view scanner discovery is a mess right now and we'll have to clean it up. Right now, though, I'm with Mikhail and have some features to get done for RC0.
Comment 11 Gerhard Schaber CLA 2007-04-12 15:47:30 EDT
Hi!

I just want to make sure that either my patch finds its way into CDT or another patch that does pretty the same thing.

Best regards
Comment 12 Doug Schaefer CLA 2007-04-12 15:48:58 EDT
I have a query to find all outstanding patches. Don't worry, we'll review all of them before 4.0 goes out the door.
Comment 13 Gerhard Schaber CLA 2007-04-19 14:52:53 EDT
Created attachment 64337 [details]
More flexible discovery patch that can handle "if", ";". wrappers, quotes, blanks
Comment 14 Gerhard Schaber CLA 2007-04-20 08:59:57 EDT
Created attachment 64427 [details]
More flexible discovery patch that can handle "if", ";". wrappers, quotes, blanks

Hi Mihail!

Could you have a look at my scanner discovery patch, please? I would appreciate it, if it was commited soon.

I know there is one little issue that we may need to address. I my patch in PerFileSICollector.applyFileDeltas I call cmd.resolveOptions. This initially propagates discovery results to the correct structures in CCommandDSC. This helps to skip the scanner info builder. We could make this optional, but I need this in any way.
Comment 15 Gerhard Schaber CLA 2007-05-07 17:12:13 EDT
Created attachment 66219 [details]
More flexible discovery patch that can handle "if", ";". wrappers, quotes, blanks

Pulled up latest patch to RC0.
Comment 16 Gerhard Schaber CLA 2007-05-16 19:17:44 EDT
Created attachment 67558 [details]
Enhanced patch with relative include paths

The new patch creates relative include paths in order to allow project renaming without losing scanner discovery info.
Comment 17 Doug Schaefer CLA 2007-05-17 16:03:06 EDT
(In reply to comment #16)
> Created an attachment (id=67558) [details]
> Enhanced patch with relative include paths
> The new patch creates relative include paths in order to allow project renaming
> without losing scanner discovery info.

Unfortunately, you are using Java 5 types in this patch. CDT is still stuck on Java 1.4. Could you fix and resubmit?
Comment 18 Gerhard Schaber CLA 2007-05-18 08:55:59 EDT
Created attachment 67792 [details]
Enhanced patch with relative include paths

Small fix for Java 1.4 compatibility. Only one method used generics in the return type.
Comment 19 Doug Schaefer CLA 2007-05-18 11:02:15 EDT
Patch applied. Thanks Gerhard.

I'm having a lot of trouble with per file scanner discovery. Hopefully this will help. But we definitely need to test it more.
Comment 20 Gerhard Schaber CLA 2007-05-18 13:10:10 EDT
Hi Doug!

Thanks for applying it. Yes, I agree, it really needs some thorough testing.

The patch solves a lot of issues, but I am afraid there are still open issues such as
- performance and memory consumption
- if-then-else statements: compiler commands in the then or else branch are not considered for scanner detection (if A then B else C delivers only scanner info for A)--rather rare scenarios
- GCCPerFileBOPConsoleParserUtility.addGenericCommandForFile2 doies not work with whitespace in paths (just in case that someone plans to use it sometime)
- all supported compiler options should be tested thoroughly, and fixed, if necessary
- check cases where the name of any compiler command occurs in a compiler path, for instance, if the compiler command is:
   c:/Programs/gcc/bin/gcc ... x.c // make sure that this is interpreted correctly
or if the compiler does not even occur:
   c:/Programs/accelerator/test.exe x.c // should not be interpreted as compiler command
Comment 21 Gerhard Schaber CLA 2007-05-18 13:15:38 EDT
regarding my last statement:
The problem in path "accelerator" might occur if a compiler "cc" is supported--a line with "accelerator" may be interpreted as compiler command in this case. It is very unlikely though.
Comment 22 Doug Schaefer CLA 2008-06-03 14:41:02 EDT
assigning
Comment 23 Doug Schaefer CLA 2008-06-03 14:41:25 EDT
done