Bug 67718 - Build Console dramatically slows down builds on Linux
Summary: Build Console dramatically slows down builds on Linux
Status: VERIFIED FIXED
Alias: None
Product: CDT
Classification: Tools
Component: cdt-core (show other bugs)
Version: 2.0   Edit
Hardware: PC Linux
: P2 major (vote)
Target Milestone: 2.0.1   Edit
Assignee: Andrew Niefer CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-06-17 16:48 EDT by Brent Nicolle CLA
Modified: 2004-10-20 17:09 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brent Nicolle CLA 2004-06-17 16:48:44 EDT
CDT 2.0.0.200406140300
Eclipse RC2
IBM JRE 1.4.2

On a RedHat Linux machine, running off a local disk,
I touch a header, and my build needs to rebuild 63 files.
From the command-line, the build takes 5.6 seconds.

From within CDT, touch the exact same header file,
the build takes 2 minutes and 15 seconds!
(Actually it's not the build, it's the console view;
ps and "time make" indicate that the compile takes its
usual short time.)

I turned off all of the Error Parsers and Scanner Config
but it made no noticeable difference.

(I do not see the same trouble on Windows.)
Comment 1 David Daoust CLA 2004-06-25 10:23:00 EDT
Is this the J9 VM?
Comment 2 David Daoust CLA 2004-06-25 10:31:18 EDT
Assigning to Andrew for investigation
Comment 3 Brent Nicolle CLA 2004-06-25 10:45:08 EDT
This was *not* the J9 VM, and was using May 25 IBM JRE.

Caution: The problem may be non-trivial to reproduce.
I was using CppTargetRTS as a project.
Comment 4 John Camelon CLA 2004-06-25 11:12:26 EDT
I'll snag this one while I wait for other responses to my last gating 
defect ... :-)
Comment 5 John Camelon CLA 2004-06-25 11:16:11 EDT
Brent, can you point me towards this project?
Comment 6 Brent Nicolle CLA 2004-06-25 12:05:26 EDT
/home/bnicolle/cdt/big_projects/CppTargetRTS.psf
Comment 7 Alain Magloire CLA 2004-06-25 14:06:50 EDT
> I'll snag this one while I wait for other responses to my last gating 
> defect ... :-)

Ok, but to let you on a little secret SWT/GTK is __really__ slow
when comparing to SWT/Win32.  I do not even want to start on SWT/motif.

When building we do some processing(error parsers, scanner discovery,
redirected to console view etc ...)

So building from eclipse will __always__ be slower then on a terminal.

Comment 8 Brent Nicolle CLA 2004-06-28 12:17:58 EDT
This is a serious problem, which reduces the usefulness of using
CDT for compiling big projects on Linux.
Comment 9 Andrew Niefer CLA 2004-06-28 15:41:10 EDT
I copy/pasted build output into a make.txt file to end up with 2000 lines of 
build output.  my makefile was then "all:  cat make.txt".
Both windows and linux were comparably slow to output the build results (on the 
order of minutes for a build operation that took on the order of seconds).
Turning off the error parsing and automated discovery parsing had no obvious 
affect on the time.
Comment 10 Andrew Niefer CLA 2004-06-29 15:34:40 EDT
The problem is that we are flushing the stream between the process and the 
console on every read ( approx every 300 bytes on my test project ).  And each 
flush creates a new Runnable to update the console UI.

My initial changes which flush only when 1024 bytes make it through the stream 
reduces the time of a 5s shell build from 1m15s to 20s in the console.

Also ErrorParserManager.checkLine(bool) is a little wastefull with the amount 
of string objects created, so we might be able to save a little time there.

However, I'm not yet comfortable enough with these changes to put them in 2.0, 
so I'm changing the target milestone to 2.0.1
Comment 11 Andrew Niefer CLA 2004-07-05 11:42:51 EDT
we are now exercising some restraint around how often we flush the pipes.
fix commited to both head and 2.0 branch.
Comment 12 David Inglis CLA 2004-07-13 21:42:57 EDT
I've checked in some further improvements here for the build console (no more
pipes, improved buffer mechanism on the ui side) some timings here with win32
and gtk show nice improvments with 'make -d', on a 1ghz athalon winxp, was
~20sec, now ~5sec, gtk (don't know the machine speed..) was 1min. 30sec, now
9-10secs using default console properties. I'll let it soak on the HEAD for now,
if no problems are found I'll move to the branch.
Comment 13 Daniel Rolf CLA 2004-07-16 11:13:48 EDT
I have problems using Sun's JRE Version 1.4.2 on Windows. It has got a flaw in 
the PipedInputStream:

1. Precondition: pipe is full.
2. WRITER writes to the pipe and "wait"s because the pipe is full.
3. READER reads from the pipe, but a "notify" to release the WRITER thread is 
missing.
4. After one second the WRITER's "wait" timeouts and the WRITER recognizes that 
the pipe is not full anymore and appends its data.

Fix: PipedInputStream must call "notifyAll" at the end of a "read" operation.

Although this is not eclipse's problem, I reopen because I think JRE 1.4.2 is 
quite common.
Comment 14 Alain Magloire CLA 2004-07-16 11:17:38 EDT
> I have problems using Sun's JRE Version 1.4.2 on Windows. It has got a flaw in 
> the PipedInputStream:

Check the head, the pipes been removed.
I doubt however that it will make it in the 2.0.1 branch
because of the volume of the changes.

But give the head a go, it probably fix your problem.


Comment 15 Daniel Rolf CLA 2004-07-16 12:02:14 EDT
Looks good now!
Comment 16 David Inglis CLA 2004-08-03 13:55:08 EDT
changing state back...
Comment 17 Brent Nicolle CLA 2004-10-20 17:09:53 EDT
Verified in 2.1
(Problem still exists in 2.0.2)
Thanks, Dave.