Bug 547271 - [Performance] Filtering and adding objects to large tables is slow
Summary: [Performance] Filtering and adding objects to large tables is slow
Status: CLOSED FIXED
Alias: None
Product: ECP
Classification: Modeling
Component: EMF Forms (show other bugs)
Version: 1.20.0   Edit
Hardware: All All
: P3 normal (vote)
Target Milestone: 1.21.0   Edit
Assignee: Christian Damus CLA
QA Contact: Eugen Neufeld CLA
URL:
Whiteboard:
Keywords: test
Depends on:
Blocks:
 
Reported: 2019-05-14 16:37 EDT by Christian Damus CLA
Modified: 2019-06-18 10:11 EDT (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Damus CLA 2019-05-14 16:37:52 EDT
The performance of table refreshes when filtering on column values and adding objects (as separate concerns, not necessarily together) degrades dramatically as the number of objects presented in the table increases (irrespective of validation problems that may be shown).  In my application, I'm seeing this in the Nebula GridViewer renderer particularly but it probably is also an issue for the SWT TableViewer renderer.

There are a few bottlenecks of particular concern.  For filtering,
  - determination of a filter match relies on updating a cell via the CellLabelProvider. This is much more expensive that is really needed for filtering, which only needs to know the text that is presented.  The cell update sets foreground and background colours (which for some reason takes something like 25% of the time) and requires creation of a fake viewer row and cell for every column of every object filtered
  - on dismissing the filter controls, the entire table is refreshed by the an inner class of the GridViewerColumnBuilder for every column.  For tables with a large number of rows and more than a few columns, this really adds up.  Only one refresh is required for the whole table

For adding rows to tables that already have a large number of rows, when the table has a detail panel in which the detail views are generated automatically by the framework, the cost in the generated view-provider of generating identical detail views for every object in the table to update the control-settings mapping is quite expensive.  As a hand-crafted view model is loaded only once and then copied as needed, the generation of these automatic view models should likewise be done only once and then copied as needed.
Comment 1 Eclipse Genie CLA 2019-05-15 08:56:06 EDT
New Gerrit change created: https://git.eclipse.org/r/142182
Comment 3 Christian Damus CLA 2019-05-16 12:58:42 EDT
(In reply to Eclipse Genie from comment #2)
> Gerrit change https://git.eclipse.org/r/142182 was merged to [develop].
Comment 4 Christian Damus CLA 2019-05-17 07:46:42 EDT
Regression testing considerations:

The most important change for filtering performance was deferment of viewer refreshes to an asynchronous non-redundant refresh.  In the table viewer this is implemented using its RunnableManager to ignore refresh requests while a refresh is already pending.  This solves the problem of every column of the table performing a refresh of the table (it only needs to be done once).  But, the asynchronous nature of this refresh can cause other consequent events to occur in a different order than previously, and test automation of the UI may need to process the event loop explicitly before making assertions (RCPTT handles this automatically).

The dynamically-generated view models employed in cases where a custom view model was not registered are now no longer generated from scratch every time they are needed.  Instead, they are cached (within a limit) per EClass and copied for each request to maintain their mutability in the editor.  During this copy step, the model's edit providers are consulted to determine which controls should be read-only, as before during the generation of those controls.  However, for models that have edit-providers that determine on a per-instance basis whether to even provide certain property sources or not, which previously would result in some controls not being generated at all, applications would probably need to define static view models with visibility rules instead on the 1.21 release.  These cases should be exceedingly rare, as it would have been a better solution to use explicit visibility rules anyways (that's why they exist) than to dynamically filter property sources in the edit providers.
Comment 5 Eclipse Genie CLA 2019-05-17 11:11:04 EDT
New Gerrit change created: https://git.eclipse.org/r/142338
Comment 7 Christian Damus CLA 2019-06-18 10:11:12 EDT
The fix is published in 1.22 M1a.