Bug 517446 - Poor scrolling performance of Terminal
Summary: Poor scrolling performance of Terminal
Status: NEW
Alias: None
Product: CDT
Classification: Tools
Component: terminal (show other bugs)
Version: 9.11.0   Edit
Hardware: PC Mac OS X
: P3 normal with 5 votes (vote)
Target Milestone: ---   Edit
Assignee: Project Inbox CLA
QA Contact: Jonah Graham CLA
URL:
Whiteboard:
Keywords: helpwanted
Depends on: 366471
Blocks:
  Show dependency tree
 
Reported: 2017-05-30 11:11 EDT by Doug Schaefer CLA
Modified: 2020-04-30 03:33 EDT (History)
10 users (show)

See Also:


Attachments
CPU sampling profile while scrolling a small terminal window (320.91 KB, image/png)
2017-06-28 01:36 EDT, Jurgen Vinju CLA
no flags Details
Another CPU profile, showing how much time is in sleep() (406.52 KB, image/png)
2017-06-28 04:02 EDT, Jurgen Vinju CLA
no flags Details
hotpath of the scrolling view (275.38 KB, image/png)
2017-11-22 11:01 EST, Davy Landman CLA
no flags Details
hotpath of the repainting (not just scrolling) (188.86 KB, image/png)
2017-11-22 11:03 EST, Davy Landman CLA
no flags Details
hotpath of scrolling to the end after a new line is printed (216.32 KB, image/png)
2017-11-22 11:04 EST, Davy Landman CLA
no flags Details
summarized hotspots view that aggregates the different paths to the fillRectangle call (96.99 KB, image/png)
2017-11-22 14:08 EST, Davy Landman CLA
no flags Details
Movie of scrolling in a tm.terminal (12.34 MB, video/quicktime)
2017-11-23 04:00 EST, Jurgen Vinju CLA
no flags Details
screen-recording of scrolling behavior of a find & cat of a bunch of files. (27.15 MB, video/quicktime)
2017-11-23 04:38 EST, Davy Landman CLA
no flags Details
mars & tm terminal 4.0 . quite a bit quicker (26.16 MB, video/quicktime)
2017-11-23 10:59 EST, Davy Landman CLA
no flags Details
4.6.3-vs-4.7.1a (18.94 MB, video/quicktime)
2018-03-29 18:39 EDT, Daniel Johnson CLA
no flags Details
4.8-integration-stack.txt (1.09 KB, text/plain)
2018-03-29 19:40 EDT, Daniel Johnson CLA
no flags Details
screenshot installing terminal milestone (202.73 KB, image/png)
2018-03-30 03:34 EDT, Martin Oberhuber CLA
no flags Details
4.8I_patch_366471-vs-4.6.3 (25.19 MB, video/quicktime)
2018-03-30 12:16 EDT, Daniel Johnson CLA
no flags Details
flamegraph produced with async-profiler while tm.terminal is scrolling; had to edit the file to remove bad characters due to a current bug in async-profiler wrt Objective C lambda expressions. (782.40 KB, image/svg+xml)
2019-11-18 15:47 EST, Jurgen Vinju CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Doug Schaefer CLA 2017-05-30 11:11:36 EDT
When I run npm in the Terminal View, npm has a bad habit of spewing lots of text of it's own when a script fails. You have to scroll above it to see the real error.

On Mac, the performance of scrolling the Terminal is very poor. It should be as fast as the Terminal view in VS.Code.
Comment 1 Martin Oberhuber CLA 2017-05-30 12:53:16 EDT
Hey Doug,

How do you scroll ... drag the scrollbar? Or use keys?

If using keys, you're bound by the key repeat rate, try System Preferences > Keyboard > Key Repeat : Fastest

Cheers,
Martin
Comment 2 Doug Schaefer CLA 2017-05-30 14:05:29 EDT
On Mac, I'm using the two finger swipe on the touchpad. I can't get the scroll bar to show up when I want either. But I assume it's the same behavior as the scroll bar.
Comment 3 Michael Scharf CLA 2017-05-31 19:04:22 EDT
I think the problem with scrolling may be that the text is drawn on a canvas and when I did that there was no two finger swipe ;-). So the question is, how does SWT canvas behave on scrolling?
Comment 4 Jurgen Vinju CLA 2017-06-27 08:25:53 EDT
We use Terminal a lot (with pleasure, so thanks!). 

However, with scrolling speed we have the same experience: too slow to be useful. 

Note that the font selected for the terminal has an enormous impact on scrolling speed. I.e. for Menlo it would be extremely slow, while Consolas will have significantly better scrolling performance. Nevertheless, even with the "fastest" fonts, scrolling back to an error is not effectively doable with the current implementation. 

Thanks for looking into this; a faster scroll would really end the discussion on which Terminal to use for our projects.
Comment 5 Martin Oberhuber CLA 2017-06-27 13:01:55 EDT
Contributions are welcome by those who can reproduce this. I don't have a Mac at the moment to have a look, but I'll happily assist anyone who's willing to try and improve things.
Comment 6 Jurgen Vinju CLA 2017-06-28 01:36:17 EDT
Created attachment 269084 [details]
CPU sampling profile while scrolling a small terminal window

This is about 32 lines of text scrolling up and down, 16 lines visible at a time, using the touchpad on my macbook pro. 

Things to note: 

* I turned on my scrollbars (always) using the System preferences, since sometimes it seems that unhiding them can cause trouble. I don't see a noticeable difference though if I turn them on again. 
* There seems no apparent bottleneck but the calls to _syncExec_ are noticeable and may block the UI thread completely.
Comment 7 Jurgen Vinju CLA 2017-06-28 01:37:10 EDT
This older discussion may be relevant: https://bugs.eclipse.org/bugs/show_bug.cgi?id=511596
Comment 8 Jurgen Vinju CLA 2017-06-28 04:02:11 EDT
Created attachment 269090 [details]
Another CPU profile, showing how much time is in sleep()
Comment 9 Davy Landman CLA 2017-11-22 10:59:35 EST
This has gotten worse since the last eclipse release (Oxygen.1a). both in TM Terminal 4.2 and the 4.3rc3.

It's now also just handling keyboard input, the characters appear about one or two seconds later than you type them.

I've done some profiling and found that in the end it's in the calls to SWT done by: org.eclipse.tm.internal.terminal.textcanvas.TextLineRenderer.drawText(org.eclipse.swt.graphics.GC, int, int, int, int, java.lang.String)

I have attached two profile screenshots of the hot paths for handling new input, and handling scrolling.

There seems to a related issue reported here: https://bugs.eclipse.org/bugs/show_bug.cgi?id=366471 Which points to adding SWT.NO_BACKGROUND on controls, but it seems to be they are talking about a different call to fillRectangle. Also, we don't experience scrolling issues in any other eclipse windows.

Macbooks with a retina screen seem to be more hurt by this laggy performance.

In our project we also have some none-PTY terminals, and they experience the same degredation in performance.

I hope you see a way to fix this, if we can help, please let us know.
Comment 10 Davy Landman CLA 2017-11-22 11:01:07 EST
Created attachment 271590 [details]
hotpath of the scrolling view
Comment 11 Davy Landman CLA 2017-11-22 11:03:05 EST
Created attachment 271591 [details]
hotpath of the repainting (not just scrolling)
Comment 12 Davy Landman CLA 2017-11-22 11:04:03 EST
Created attachment 271592 [details]
hotpath of scrolling to the end after a new line is printed
Comment 13 Martin Oberhuber CLA 2017-11-22 13:11:52 EST
So I have installed the Eclipse C/C++ package Oxygen.1a for OSX from here:
https://www.eclipse.org/downloads/packages/eclipse-ide-cc-developers/oxygen1a
My machine is a 13-inch Retina MBP 2017 running OSX 10.12.6 Sierra.

This is what I observe:
- printing text in the terminal is lightning fast, even if it scrolls. Testcase:
  find workspace | xargs cat
- scrolling with the trackpad is slow, but nowhere near the "2 seconds per line".
  For me, it scrolls approx 5 lines per second this way.
- If I grab the handle on the scrollbar with the mouse and drag it, it's super fast.
  I can scroll approx 500 lines per second easily.

To me, this feels more like an SWT problem than a TM Terminal problem.
Adding Alex on CC, @Alex what do you think?
Comment 14 Alexander Kurtakov CLA 2017-11-22 13:15:07 EST
Smth smells fishy in SWT but my OSX knowledge is zero. Lakshmi should know more.
Comment 15 Davy Landman CLA 2017-11-22 14:08:08 EST
Created attachment 271595 [details]
summarized hotspots view that aggregates the different paths to the fillRectangle call

Just a last screenshot from the profiler, with a backtrace on the hotspots in the profile.
Comment 16 Davy Landman CLA 2017-11-22 14:40:17 EST
@Martin, interestingly, in that do deployment, and also when installing the update rc3 in a normal eclipse, I don't see the scrolling problems. I'll test with the other users to see if they can confirm.

This might indicate our plugin release has a wrong version of swt in the bundle. I'll starting investigating if this is the case.
Comment 17 Jurgen Vinju CLA 2017-11-23 04:00:16 EST
Created attachment 271603 [details]
Movie of scrolling in a tm.terminal

This is on my Mac with the following (hopefully relevant) versions:

* oxygen 1a, release (4.7.1a)
* vm: jdk1.8.0_60.jdk
* -Dorg.eclipse.swt.internal.carbon.smallFonts
* tm.terminal 4.3.0 (same behavior with 4.2.0 which I also tried) 

I also downloaded the CDT version of oxygen 1a with no observable difference. 

Next to this I observed that the wider the terminal and the longer the sentences printed on each line, the slower scrolling is and also typing the next command is slower if there is more text on the current screen. It's a lot slower than in the attached movie with more text to print even.
Comment 18 Davy Landman CLA 2017-11-23 04:36:59 EST
So my earlier reports were wrong. On the CDT release of oxygen.a1 I still see the big slowdown. Yesterday evening I was working on my small screen laptop (without retina). When I switch to a larger external screen it's back as well. 

I've also captured a video that shows the effect of increasing the size of the terminal window.
Comment 19 Davy Landman CLA 2017-11-23 04:38:05 EST
Created attachment 271605 [details]
screen-recording of scrolling behavior of a find & cat of a bunch of files.
Comment 20 Lakshmi P Shanmugam CLA 2017-11-23 09:02:16 EST
(In reply to Davy Landman from comment #18)
> So my earlier reports were wrong. On the CDT release of oxygen.a1 I still
> see the big slowdown. Yesterday evening I was working on my small screen
> laptop (without retina). When I switch to a larger external screen it's back
> as well. 
> 
Is this a regression? Was it scrolling faster in 4.6 or earlier versions?
Comment 21 Davy Landman CLA 2017-11-23 10:41:33 EST
(In reply to Lakshmi Shanmugam from comment #20)
> (In reply to Davy Landman from comment #18)
> > So my earlier reports were wrong. On the CDT release of oxygen.a1 I still
> > see the big slowdown. Yesterday evening I was working on my small screen
> > laptop (without retina). When I switch to a larger external screen it's back
> > as well. 
> > 
> Is this a regression? Was it scrolling faster in 4.6 or earlier versions?

it was referring to comment #16. 

But indeed, on older version of eclipse, it was a lot quicker. (for example 4.4 and 4.5). But then again our screens were smaller back then.

I could try with a old mars edition and see what happens.
Comment 22 Davy Landman CLA 2017-11-23 10:59:23 EST
Created attachment 271613 [details]
mars & tm terminal 4.0 . quite a bit quicker

I've gone back to mars and installed TM Terminal 4.0, and it's quite a bit more quicker in scrolling. Handling keyboard input on full screen is however a bit on the slowish side.
Comment 23 Daniel Johnson CLA 2018-03-29 18:39:11 EDT
Created attachment 273368 [details]
4.6.3-vs-4.7.1a

I don't know that I can add anything that isn't already known, but just want to add my vote to this issue. This is a very noticeable regression coming from 4.6.3 up to 4.7.1a. I tested on 4.7.1a, 4.7.3, and 4.8M6 with the same delayed text entry and very slow scrolling behavior.

Attaching a video of the difference, with 4.6.3 on the left, and 4.7.1a on the right.
Comment 24 Daniel Johnson CLA 2018-03-29 19:40:34 EDT
Created attachment 273369 [details]
4.8-integration-stack.txt

FYI, I tested out the latest 4.8.0 integration build after the fix provided by Lakshmi as part of #366471. It does seem to return scrolling behavior back to 4.6.3 speeds.

However, with the latest 4.8.0 integration build, and the latest TM plugins from http://download.eclipse.org/tm/updates/4.2/, I was not able to create a new connection from the "Terminal" view (stack trace attached). So to test Lakshmi's changes I actually installed the deprecated RSE "Terminals" view plugin. Is there an integration update site of TM I should be testing with?
Comment 25 Martin Oberhuber CLA 2018-03-30 03:34:28 EDT
Created attachment 273371 [details]
screenshot installing terminal milestone

Hi Daniel,

You can get the Photon Terminal Milestones via Help > Install New Software with
http://download.eclipse.org/tm/terminal/updates/4.4milestones

Make sure you keep the "Contact all update sites during install..." enabled, since the Local Terminal depends on the Eclipse CDT providing the PTY binding. See attached screenshot. Thanks to the checkbox, you'll get org.eclipse.cdt.native macos bits from the Photon simrel repository. I've just tested with the I20180327-2000 SDK and it works fine for me.

If you ever want a nightly build, you'll get it from
http://download.eclipse.org/tm/terminal/builds/development/nightly
or Jenkins at https://ci.eclipse.org/tm/view/terminal/ (use terminal-master).
Comment 26 Martin Oberhuber CLA 2018-03-30 03:35:43 EDT
Marking "depends on" bug 366471 since it looks like the issue is in SWT.
Comment 27 Daniel Johnson CLA 2018-03-30 12:16:00 EDT
Created attachment 273387 [details]
4.8I_patch_366471-vs-4.6.3

Hi Martin, A few comments 

1) Thanks for the links, I was struggling to find those details on the TM Downloads page or TM Developer Wiki pages.

2) The exception I reported was simply because I didn't have any of the connectors installed (oops).  The call to LauncherDelegateManager.getInstance().getLauncherDelegates(false); (line 420 of LaunchTerminalSettingsDialog), was returning only the 'Streams Terminal', which the SettingsDialog filters out. Since the 'terminals' combo did not have any items, it was hitting this exception. Once I installed a connector all was well.

3) I spoke too soon, even with fixes from 366471, there is no noticeable change from 4.7.x.
Comment 28 Jurgen Vinju CLA 2019-11-18 15:47:27 EST
Created attachment 280683 [details]
flamegraph produced with async-profiler while tm.terminal is scrolling; had to edit the file to remove bad characters due to a current bug in async-profiler wrt Objective C lambda expressions.
Comment 29 Jurgen Vinju CLA 2019-11-18 15:53:33 EST
produced a flamegraph using async-profiler. Interestingly one "bottleneck" seems to be the call to `gc.fillRectangle(xx,y,fStyleMap.getFontWidth(),fStyleMap.getFontHeight());` in `TextLineRenderer::drawText` on line 131.

Eventually this call to fillRectangle (according to the profile) does some kinds of memory allocation under the hood. Is it not strange to see malloc calls as an effect of filling the rectangle under a line of text? Perhaps this is something to look into?
Comment 30 Yurii Kartsev CLA 2020-04-29 18:51:02 EDT
I would like to report the same issue in Eclipse 2020-03 (4.15.0) (build ID 20200313-1211) and OS X 10.15.3. I did not want to add another ticket for that, therefore commenting here.

I did not know that Terminal can be used in Eclipse, always used a separate app for that (native OS X terminal). But today when I figured I can use one in Eclipse, I thought it may be very useful. Although, in less than a minute I had to scroll and oh my God how slow it is! I'm also using "swipe" with two fingers to scroll. I was thinking that there may be an option to change scrolling speed in terminal in Eclipse, but I did not find one.

Would be glad to help with what I can if needed. Obviously, not a blocker, but very nice to have.

Thank you in advance!
Comment 31 Pierre-Yves Bigourdan CLA 2020-04-30 03:33:04 EDT
Moving over to CDT which now hosts the terminal project.