Bug 548982 - [GTK] Optimize bulk inserting items to Tree
Summary: [GTK] Optimize bulk inserting items to Tree
Status: VERIFIED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: SWT (show other bugs)
Version: 4.12   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 4.13 M1   Edit
Assignee: Alexandr Miloslavskiy CLA
QA Contact: Eric Williams CLA
URL:
Whiteboard:
Keywords: triaged
Depends on:
Blocks:
 
Reported: 2019-07-04 12:38 EDT by Alexandr Miloslavskiy CLA
Modified: 2021-09-02 16:39 EDT (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexandr Miloslavskiy CLA 2019-07-04 12:38:36 EDT
I will provide a patch soon.
Comment 1 Eclipse Genie CLA 2019-07-04 13:12:09 EDT
New Gerrit change created: https://git.eclipse.org/r/145490
Comment 2 Alexandr Miloslavskiy CLA 2019-07-04 13:14:02 EDT
Improvement using tests from snippet added in patch:

CREATE_REVERSE_ORDER=true
1376ms -> 535ms

CREATE_CTOR_2PARAM=true
4842ms -> 4108ms

CREATE_REVERSE_ORDER is the most important here.
Comment 3 Alexandr Miloslavskiy CLA 2019-07-04 13:15:52 EDT
Rationale: In our software (SmartGit) we show git tags in UI. Some users have repositories with very big number of tags, one user reported 21000.
Comment 4 Eric Williams CLA 2019-07-04 15:09:28 EDT
With CREATE_REVERSE_ORDER set to true, the time on my machine is ~280ms. This is down from ~4500, very nice!
Comment 6 Eric Williams CLA 2019-07-08 14:40:13 EDT
Thanks for the patch Alexandr, this is a nice improvement. Can the same performance gain be made in Table?


(In reply to Eclipse Genie from comment #5)
> Gerrit change https://git.eclipse.org/r/145490 was merged to [master].
> Commit:
> http://git.eclipse.org/c/platform/eclipse.platform.swt.git/commit/
> ?id=de95eb78d372a16b034c585d679aa7cfffc17c4d

In master now.
Comment 7 Alexandr Miloslavskiy CLA 2019-07-09 06:59:55 EDT
Is there a specific problem (or even a reported bug) with Table you have in mind?
Comment 8 Eric Williams CLA 2019-07-09 08:16:51 EDT
(In reply to Alexandr Miloslavskiy from comment #7)
> Is there a specific problem (or even a reported bug) with Table you have in
> mind?

No, but Table uses the same GTK internals and in SWT the code is quite similar, so a lot of Tree fixes apply to Table as well. It's not urgent, I mention it only because if your product also uses Tables anywhere else then it's likely this optimization will apply there as well.
Comment 9 Eric Williams CLA 2019-07-10 11:26:51 EDT
Verified in I20190710-0610.
Comment 10 Carsten Hammer CLA 2020-07-13 13:46:04 EDT
What we have currently I think is no real bulk insert. It is still a one row at a time approach.
According to http://scentric.net/tutorial/sec-treemodel-add-rows.html
I should do 3 things on bulk insert:

3.3.3. Speed Issues when Adding a Lot of Rows

A common scenario is that a model needs to be filled with a lot of rows at some point, either at start-up, or when some file is opened. An equally common scenario is that this takes an awfully long time even on powerful machines once the model contains more than a couple of thousand rows, with an exponentially decreasing rate of insertion. As already pointed out above, writing a custom model might be the best thing to do in this case. Nevertheless, there are some things you can do to work around this problem and speed things up a bit even with the stock Gtk+ models:

Firstly, you should detach your list store or tree store from the tree view before doing your mass insertions, then do your insertions, and only connect your store to the tree view again when you are done with your insertions. Like this:


  ...

  model = gtk_tree_view_get_model(GTK_TREE_VIEW(view));

  g_object_ref(model); /* Make sure the model stays with us after the tree view unrefs it */

  gtk_tree_view_set_model(GTK_TREE_VIEW(view), NULL); /* Detach model from view */

  ... insert a couple of thousand rows ...

  gtk_tree_view_set_model(GTK_TREE_VIEW(view), model); /* Re-attach model to view */

  g_object_unref(model);

  ...

Secondly, you should make sure that sorting is disabled while you are doing your mass insertions, otherwise your store might be resorted after each and every single row insertion, which is going to be everything but fast.

Thirdly, you should not keep around a lot of tree row references if you have so many rows, because with each insertion (or removal) every single tree row reference will check whether its path needs to be updated or not. 

I do not see any code in swt working this way. Did anybody try this out? Is it still true or not related to latest gtk versions?
Comment 11 Alexandr Miloslavskiy CLA 2020-07-15 14:54:43 EDT
My experience with GTK is that things usually don't work the way I expect :)

You're welcome to give it a try. If that gives a good performance improvement, that would definitely be nice.