Bug 519164 - [GTK] Bad performance of expanding trees on linux
Summary: [GTK] Bad performance of expanding trees on linux
Status: NEW
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.6   Edit
Hardware: PC Linux
: P3 normal with 1 vote (vote)
Target Milestone: ---   Edit
Assignee: Platform-SWT-Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance, triaged
: 564552 (view as bug list)
Depends on:
Blocks: 349869 519166
  Show dependency tree
 
Reported: 2017-07-04 09:51 EDT by Michał Rapacz CLA
Modified: 2020-11-23 09:05 EST (History)
9 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Rapacz CLA 2017-07-04 09:51:54 EDT
Expanding large trees on linux takes very long time and it is about 10x slower than on Windows. For example, expanding tree node with 20k children in a tree viewer with 10 columns takes ~8 seconds on Windows 7, while on linux it takes ~2 minutes (same machine). Similar results were observed on multiple linux machines running RedHat/Fedora/Debian. Switching between gtk2/gtk3 does not make any noticeable difference.

Here is some sample code:

import java.util.stream.Collectors;
import java.util.stream.IntStream;

import org.eclipse.jface.layout.GridDataFactory;
import org.eclipse.jface.layout.TreeColumnLayout;
import org.eclipse.jface.viewers.ColumnLabelProvider;
import org.eclipse.jface.viewers.ColumnWeightData;
import org.eclipse.jface.viewers.ITreeContentProvider;
import org.eclipse.jface.viewers.TreeViewer;
import org.eclipse.jface.viewers.TreeViewerColumn;
import org.eclipse.jface.viewers.Viewer;
import org.eclipse.swt.SWT;
import org.eclipse.swt.widgets.Composite;
import org.eclipse.swt.widgets.Display;
import org.eclipse.swt.widgets.Shell;
import org.eclipse.swt.widgets.Tree;
import org.eclipse.swt.widgets.TreeColumn;

public class Performance {

    public static void main(final String[] args) {
        final Display display = new Display();
        final Shell shell = new Shell(display);

        final Composite viewerComposite = new Composite(shell, SWT.NONE);
        final Tree tree = new Tree(viewerComposite, SWT.MULTI | SWT.H_SCROLL | SWT.V_SCROLL);
        final TreeViewer viewer = new TreeViewer(tree);

        final TreeColumnLayout columnLayout = new TreeColumnLayout();
        viewerComposite.setLayout(columnLayout);
        for (int i = 0; i < 10; i++) {
            final TreeColumn column = new TreeColumn(tree, SWT.NONE);
            column.setText(Integer.toString(i));
            columnLayout.setColumnData(column, new ColumnWeightData(1));
            final TreeViewerColumn col = new TreeViewerColumn(viewer, column);
            col.setLabelProvider(new ColumnLabelProvider() {

                @Override
                public String getText(final Object element) {
                    return element.toString();
                }
            });
        }

        GridDataFactory.fillDefaults().grab(true, true).hint(800, 600).applyTo(viewerComposite);

        viewer.setContentProvider(new ITreeContentProvider() {

            @Override
            public Object[] getElements(final Object inputElement) {
                if ("root".equals(inputElement.toString())) {
                    return new String[] { "rootWithManyChildren" };
                } else {
                    return getChildren(inputElement);
                }
            }

            @Override
            public Object[] getChildren(final Object parentElement) {
                return IntStream.range(0, 20000)
                        .boxed()
                        .map(i -> i.toString())
                        .collect(Collectors.toList())
                        .toArray(new String[0]);
            }

            @Override
            public Object getParent(final Object element) {
                return null;
            }

            @Override
            public boolean hasChildren(final Object element) {
                return true;
            }

            @Override
            public void dispose() {
                // do nothing
            }

            @Override
            public void inputChanged(final Viewer viewer, final Object oldInput, final Object newInput) {
                // do nothing
            }

        });
        viewer.getTree().setHeaderVisible(true);
        viewer.setInput("root");

        viewerComposite.setBounds(0, 0, 800, 600);
        shell.setSize(820, 620);
        shell.open();
        while (!shell.isDisposed()) {
            if (!display.readAndDispatch()) {
                display.sleep();
            }
        }
        display.dispose();
    }
}

The problem can be observed when expanding the root node in the sample app.

Is there any reason for such huge performance difference between windows and linux? Is there a way to improve it so expanding large trees under gtk would take some more reasonable time?
Comment 1 Eric Williams CLA 2018-08-16 11:52:45 EDT
Still reproducible.
Comment 2 Thomas Wolf CLA 2018-08-17 09:02:26 EDT
Some quick CPU sampling with VisualVM shows that most of the time is spent inside org.eclipse.swt.internal.gtk.OS._gtk_tree_store_set[native], called via org.eclipse.jface.viewers.ViewerCell.setText(), setForeground(), and setBackground().

There's clearly something non-linear going on somewhere.

children   elapsed time
2000       1s
4000       3.5s
8000       14s
12000      30s
16000      58s
Comment 3 Thomas Wolf CLA 2018-08-17 09:18:41 EDT
See also bug 349869 comment 18.
Comment 4 Eric Williams CLA 2018-08-17 09:47:51 EDT
We are aware of the Table/Tree performance issues, once most GTK3 critical bugs are stabilized we will target performance bugs.
Comment 5 Lars Vogel CLA 2019-02-28 06:01:11 EST
AFAIK Xi did a lot of performance improvements in the SWT GTK tree implementation. 

Xi, is this issue still open?
Comment 6 Xi Yan CLA 2019-02-28 10:01:44 EST
(In reply to Lars Vogel from comment #5)
> AFAIK Xi did a lot of performance improvements in the SWT GTK tree
> implementation. 
> 
> Xi, is this issue still open?

This is still an issue. Expanding takes about 50 seconds which is still significantly slower that the ~8 on Win.
Comment 7 Eclipse Genie CLA 2019-02-28 15:19:54 EST
New Gerrit change created: https://git.eclipse.org/r/137829
Comment 8 Xi Yan CLA 2019-02-28 15:29:25 EST
(In reply to Eclipse Genie from comment #7)
> New Gerrit change created: https://git.eclipse.org/r/137829

Profiling shows that most of the time is spent in Tree#getId called from Tree#createItem. This patch removes the while loop in getId method, which slightly improves the performance from ~58s to ~45s with the snippet. However, there is still something deeper going on that is slowing down the performance, and likely due to the memory overhead of caching large number of TreeItems.
Comment 9 Lars Vogel CLA 2020-06-10 09:34:45 EDT
Paul, I noticed that you make impressive changes in SWT GTK. Can you have a look at this bug too?
Comment 10 Soraphol (Paul) Damrongpiriyapong CLA 2020-06-10 15:52:26 EDT
Sure thing. Currently working on scaling bugs right now, but will pick this one up as well.
Comment 11 Soraphol (Paul) Damrongpiriyapong CLA 2020-06-19 15:32:03 EDT
After some testing, I have found that the part that causes the issue has to do with gtk_tree_store_append. Trying with gtk_tree_store_prepend makes a huge difference in performance. However the problem is that the items will then be added in reversed ordered. 

Possible solutions:

- In Jface, when adding a list of items, we reverse it before & use prepend by default when creating items. However, this will change the current behavior of createItem(TreeItem, long, int).

- Another solution, one I have just tried, is to use gtk_tree_store_insert_after & use a cached sibling. So far, the performance has not changed, I believe it has to do with gtk_tree_model_iter_next which has the same performance issues. I am looking for alternatives to using append.
Comment 12 Soraphol (Paul) Damrongpiriyapong CLA 2020-06-19 16:54:42 EDT
To add, I think that the JFace changes is the best option, since that is the one in control of the entire list of items. Whereas the Tree or TreeItem classes can't do much. From what I have seen anything to do with the GtkIter slows it down dramatically.
here is a snippet of what I tried:
boolean parentHasExistingChildren = parentIter != 0 && GTK.gtk_tree_model_iter_has_child(modelHandle, parentIter);

		if (parentHasExistingChildren) {
			if (!cachedSiblings.containsKey(parentIter)) {
				long newCachedSibling = OS.g_malloc (GTK.GtkTreeIter_sizeof ());
				GTK.gtk_tree_model_iter_nth_child(modelHandle, newCachedSibling, parentIter, 0);
				cachedSiblings.put(parentIter, newCachedSibling);
			}

			Long cachedSibling = cachedSiblings.get(parentIter);
			GTK.gtk_tree_store_insert_after(modelHandle, item.handle, parentIter, cachedSibling.longValue());

			GTK.gtk_tree_model_iter_next(modelHandle, cachedSibling.longValue()); //slows things down I believe
		} else {
			GTK.gtk_tree_store_prepend (modelHandle, item.handle, parentIter);
		}
Comment 13 Soraphol (Paul) Damrongpiriyapong CLA 2020-06-22 15:27:10 EDT
Changes made to JFace
https://git.eclipse.org/r/#/c/165318/
Comment 14 Soraphol (Paul) Damrongpiriyapong CLA 2020-07-31 15:17:53 EDT
*** Bug 564552 has been marked as a duplicate of this bug. ***
Comment 15 Eclipse Genie CLA 2020-07-31 16:23:02 EDT
New Gerrit change created: https://git.eclipse.org/r/c/platform/eclipse.platform.ui/+/167135
Comment 16 Carsten Hammer CLA 2020-08-01 05:55:04 EDT
@Paul,
did you check out https://bugs.eclipse.org/bugs/show_bug.cgi?id=548982#c10?
Not sure if it helps but as I currently have no build environment for the native part I cannot check myself currently.
Comment 17 Michael Haubenwallner CLA 2020-11-23 09:05:29 EST
I've discovered this problem while trying to fix bug#564569:
Especially for the virtual tree when using a lazy data provider I would have expected that expanding a subtree does create the visible GTK items only, not all of them.

Why is there actually need to create invisible GTK items at all?