Community
Participate
Working Groups
As mention in bug 374441 comment 41, there are 41 test suites identified as having "performance targets". This I determined by looking in "test.properties", which is created during a build, by a custom CBI maven task (that, someone wrote for us ... Thanh? Igor?) ... and it finds 41. I'll be going through the list, and see which ones work, and which ones don't. I made my first major discovery tonight ... so thought I'd document it here ... and use this bug to document other findings. The major finding was that 'org.eclipse.ant.tests.core' is one of the suite identified as having a performance target, yet in preliminary runs, it seemed that "nothing ever happened", so I'm thinking, great, some thing is broken. But, finally looked at the test.xml for that 'org.eclipse.ant.tests.core' bundle, and saw <!-- This target runs the performance test suites. --> <target name="performance"> </target> That is, it has a performance target, that is completely empty! Not sure if this was changed at some point in history, or left in as a reminder to add some performance tests some day. But, bottom line, I think we should not allow empty performance targets. (Not sure if there are others, or not) but, this sort of thing is nothing but confusing unless someone looks closely, and that's a waste of time.
Another "finding" was that several of the "jdt" tests run out of memory when its set to 500m. I don't know what I was thinking ... but, will set to 2G to avoid "minor" memory issues, but suspect that's about as high as we want to go?
By manually "inspecting" the test.xml file, determined there are 19 other tests that have empty performance targets (for 20 total, leaving 21 that do have performance targets). [No wonder I thought so many were not working :) ] eclipse.jdt.core/org.eclipse.jdt.apt.pluggable.tests/test.xml eclipse.jdt.core/org.eclipse.jdt.compiler.apt.tests/test.xml eclipse.jdt.core/org.eclipse.jdt.compiler.tool.tests/test.xml eclipse.jdt.core/org.eclipse.jdt.core.tests.builder/test.xml eclipse.jdt.core/org.eclipse.jdt.core.tests.compiler/test.xml eclipse.jdt.core/org.eclipse.jdt.core.tests.model/test.xml eclipse.jdt.ui/org.eclipse.ltk.core.refactoring.tests eclipse.jdt.ui/org.eclipse.ltk.ui.refactoring.tests eclipse.pde.build/org.eclipse.pde.build.tests eclipse.pde.ui/ds/org.eclipse.pde.ds.tests eclipse.platform.runtime/tests/org.eclipse.core.expressions.tests/test.xml eclipse.platform.team/tests/org.eclipse.core.tests.net/test.xml eclipse.platform.team/tests/org.eclipse.team.tests.core eclipse.platform.text/org.eclipse.core.filebuffers.tests/test.xml eclipse.platform.text/org.eclipse.jface.text.tests eclipse.platform.text/org.eclipse.search.tests eclipse.platform.text/org.eclipse.text.tests eclipse.platform.text/org.eclipse.ui.editors.tests eclipse.platform.text/org.eclipse.ui.workbench.texteditor.tests eclipse.platform.ua/org.eclipse.ua.tests.doc
Bugs open on components with empty targets. These "bugs" don't prevent progress from being made, but, it makes the results hard to interpret the console comes back and says, basically "no results found for 'xyx performance tests' " ... impossible to distinguish that from a test that just "did run" for other reasons. My guess is historically a lot of the tools that "analyze" the data just ignores those cases, so do not so up as "failed" (or "DNF") by the time it gets to that point ... but for someone new to the performance tests, it's very confusing.
My counts are adding up, so I may have "lost" one ... but sure if "empty" one, or one with a valid target, but the list of tests I currently have to run, are the following 19: org.eclipse.ant.tests.ui org.eclipse.compare.tests org.eclipse.core.tests.runtime org.eclipse.core.tests.resources org.eclipse.equinox.p2.tests.ui org.eclipse.jdt.core.tests.performance org.eclipse.jdt.debug.tests org.eclipse.jdt.text.tests org.eclipse.jdt.ui.tests org.eclipse.jdt.ui.tests.refactoring org.eclipse.osgi.tests org.eclipse.pde.api.tools.tests org.eclipse.pde.ui.tests org.eclipse.swt.tests org.eclipse.team.tests.cvs.core org.eclipse.ua.tests org.eclipse.ui.tests.forms org.eclipse.ui.tests.performance org.eclipse.ui.tests.rcp Let me know if anyone sees any obvious ones missing. (Though, eventually we'll find it, when I no longer have to "manually" manage the list, but have accurate targets).
(In reply to David Williams from comment #0) > But, finally looked at the test.xml for that 'org.eclipse.ant.tests.core' > bundle, and saw > > <!-- This target runs the performance test suites. --> > <target name="performance"> > </target> Fixed with http://git.eclipse.org/c/platform/eclipse.platform.git/commit/?id=9769a843ecc2089a3ad55b149402893e4d5a5ee8
All dependent bugs are fixed.
Confirmed there are 19 tests bundles (in test.properties) now that "empty ones' have been removed: org.eclipse.ant.tests.ui org.eclipse.compare.tests org.eclipse.core.tests.resources org.eclipse.core.tests.runtime org.eclipse.equinox.p2.tests.ui org.eclipse.jdt.core.tests.performance org.eclipse.jdt.debug.tests org.eclipse.jdt.text.tests org.eclipse.jdt.ui.tests org.eclipse.jdt.ui.tests.refactoring org.eclipse.osgi.tests org.eclipse.pde.api.tools.tests org.eclipse.pde.ui.tests org.eclipse.swt.tests org.eclipse.team.tests.cvs.core org.eclipse.ua.tests org.eclipse.ui.tests.forms org.eclipse.ui.tests.performance org.eclipse.ui.tests.rcp
