Bug 90815 - [content type] performance tests failing consistently
Summary: [content type] performance tests failing consistently
Status: RESOLVED FIXED
Alias: None
Product: Platform
Classification: Eclipse Project
Component: Runtime (show other bugs)
Version: 3.1   Edit
Hardware: PC Windows XP
: P3 normal (vote)
Target Milestone: 3.1 M7   Edit
Assignee: Rafael Chaves CLA
QA Contact:
URL:
Whiteboard:
Keywords: performance
Depends on:
Blocks:
 
Reported: 2005-04-08 11:21 EDT by Rafael Chaves CLA
Modified: 2005-05-09 19:21 EDT (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rafael Chaves CLA 2005-04-08 11:21:25 EDT
The recently added performance tests (see bug 57137 comment 6) are failing
consistently.

Need to investigate whether is an actual regression or a problem with the tests
themselves.
Comment 1 Rafael Chaves CLA 2005-04-08 11:27:50 EDT
See also bug 88208, where a problem with the collection of performance data in
those tests was fixed.
Comment 2 Rafael Chaves CLA 2005-04-13 13:12:34 EDT
Improved loadCatalog. It is better than 3.0 now. We used to check preferences
for every content type being created, now we do it once after creating all
content types, and only for content types that actually do have user settings.
Comment 3 Rafael Chaves CLA 2005-04-13 17:13:31 EDT
There was a performance regression introduced by the fix to bug 67841.
LazyInputStream/Reader was capturing IOExceptions in methods that run very
often. This made describers run slower. Moved the try/catch block closer to the
source of exceptions (#ensureAvailable), which runs much less often.
This affects only testContentMatching.
Comment 4 Rafael Chaves CLA 2005-04-13 17:21:17 EDT
There was a big performance hit caused by the way the extension registry is
implemented now. ContentType#getDescriber used to query its configuration
element every time. The performance characteristics of the registry API changed
a lot since 3.0 so querying a configuration element became expensive (obtaining
the describer would almost take the same cost of describing the contents).
Changed ContentType#getDescriber to query the configuration element only once.
Comment 5 Rafael Chaves CLA 2005-04-13 19:03:41 EDT
Improved performance for testIsKindOf as well. Will wait for next performance
results before closing this.
Comment 6 Rafael Chaves CLA 2005-05-02 11:53:01 EDT
#loadCatalog is the only test failing now. It is not a critical scenario
(happens only once per session - if plug-ins are not installed/removed). The
probable reason for being slower is that the extension registry might be slower
now. All other ContentTypeManager tests are passing.
Comment 7 Rafael Chaves CLA 2005-05-08 22:10:47 EDT
John, I forgot to ask you again: what are we supposed to do with tests that are
expected to fail (because they are acknowlegedly slower than in 3.0)?
Comment 8 John Arthorne CLA 2005-05-09 15:33:16 EDT
There is API in the performance test suite for adding a regression comment:

Performance.getDefault().setComment(
				fPerformanceMeter, 
				Performance.EXPLAINS_DEGRADATION_COMMENT, 
				"This regression is expected because ...");
Comment 9 Rafael Chaves CLA 2005-05-09 19:21:13 EDT
Ok, got it.