Community
Participate
Working Groups
Looking at results for builds N20100814-2000 and M20100811-0800, it appears that this is not the correct baseline which is used while generating the results. E.g. for the maintenance build, the generated HTML pages show the following title: Performance of M20100811-0800 relative to R-3.5-200906111540 (201007261800) Although in the console log, it's said: + no baseline specified => use last one: R-3.5-200906111540_201008091800
Increase the severity as this gives to incorrect performance results and may lead to wrong conclusions during the verification!
(In reply to comment #1) > Increase the severity as this gives to incorrect performance results and may > lead to wrong conclusions during the verification! I would have said "inaccurate results" instead of "incorrect results". But this issue must still be fixed asap...
I've temporarily fixed results for builds N20100814-2000 and M20100811-0800 by generating them manually. It looks like the problem is that last baseline is not added to local files, hence cannot be used while generating the results. I'll investigate further...
As the verification is done using a specific tool and not the generated HTML pages, I think the severity can be reduced and the bug deferred to next milestone.
Ping.
Sorry, not for this milestone... :-( Maybe Satyam could have a look for next one?
(In reply to comment #4) > As the verification is done using a specific tool and not the generated HTML > pages, I think the severity can be reduced and the bug deferred to next > milestone. Increasing severity again: most people look at the web pages and don't use the tool.
I do see that there is a lag in the baseline that is used, because of which the tool and the generated html files sometimes use a different baseline. Is that the problem or am I missing anything?
Frédéric, can you give Satyam some hints?
(In reply to comment #8) > I do see that there is a lag in the baseline that is used, because of which the > tool and the generated html files sometimes use a different baseline. Is that > the problem or am I missing anything? Yes, that's the problem. I seemed that the tool was using the correct baseline although the generation used an older one. The first step is to try to generate the results using the same command line as the one used in the ant script and see which baseline is used during the generation. I'll attach a launch config I used to generate the HTML results manually from my workspace... If the used baseline is not the same, then it's easy to use the launch config in debug mode (after having changed the directories, of course) to track the problem down. HTH
Created attachment 183287 [details] Sample of launch config to generate HTML results similarily as the Releng Ant script does Satyam, feel free to contact me if you still have some troubles to reproduce or debug this issue...
It seems the performance results generation is failing with the following error message. Is this related to this bug !ENTRY org.eclipse.test.performance 4 1 2010-11-17 09:18:51.016 !MESSAGE Execution failed due to a distribution protocol error that caused deallocation of the conversation. The command requested could not be completed because of a permanent error condition detected at the target system. !ENTRY org.eclipse.test.performance 4 1 2010-11-17 09:18:51.024 !MESSAGE Execution failed due to a distribution protocol error that caused deallocation of the conversation. The command requested could not be completed because of a permanent error condition detected at the target system. !ENTRY org.eclipse.osgi 2 0 2010-11-17 09:18:51.208 !MESSAGE One or more bundles are not resolved because the following root constraints are not resolved: !SUBENTRY 1 org.eclipse.osgi 2 0 2010-11-17 09:18:51.209 !MESSAGE Bundle update@plugins/org.eclipse.equinox.p2.ui.sdk_1.0.200.v20100927-1600.jar was not resolved. !SUBENTRY 2 org.eclipse.equinox.p2.ui.sdk 2 0 2010-11-17 09:18:51.209 !MESSAGE Missing required bundle org.eclipse.compare_0.0.0. !ENTRY org.eclipse.osgi 2 0 2010-11-17 09:18:51.220 !MESSAGE The following is a complete list of bundles which are not resolved, see the prior log entry for the root cause if it exists: !SUBENTRY 1 org.eclipse.osgi 2 0 2010-11-17 09:18:51.220 !MESSAGE Bundle org.eclipse.equinox.launcher.win32.win32.x86_1.1.100.v20101004 [49] was not resolved. !SUBENTRY 1 org.eclipse.osgi 2 0 2010-11-17 09:18:51.221 !MESSAGE Bundle org.eclipse.swt.win32.win32.x86_3.7.0.v3712b [69] was not resolved. !SUBENTRY 1 org.eclipse.osgi 2 0 2010-11-17 09:18:51.221 !MESSAGE Bundle org.eclipse.equinox.p2.ui.sdk_1.0.200.v20100927-1600 [113] was not resolved. !SUBENTRY 2 org.eclipse.equinox.p2.ui.sdk 2 0 2010-11-17 09:18:51.222 !MESSAGE Missing required bundle org.eclipse.compare_0.0.0. !ENTRY org.eclipse.osgi 4 0 2010-11-17 09:18:51.222 !MESSAGE Application error !STACK 1 java.lang.NullPointerException at org.eclipse.test.internal.performance.results.db.DB_Results.getLastBaselineBuild(DB_Results.java:669) at org.eclipse.test.internal.performance.results.db.PerformanceResults.setDefaults(PerformanceResults.java:766) at org.eclipse.test.internal.performance.results.db.PerformanceResults.<init>(PerformanceResults.java:108) at org.eclipse.test.performance.ui.GenerateResults.setPerformanceResults(GenerateResults.java:1041) at org.eclipse.test.performance.ui.GenerateResults.parse(GenerateResults.java:475) at org.eclipse.test.performance.ui.GenerateResults.run(GenerateResults.java:823) at org.eclipse.test.performance.ui.Main.start(Main.java:40) at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110) at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369) at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:621) at org.eclipse.equinox.launcher.Main.basicRun(Main.java:576) at org.eclipse.equinox.launcher.Main.run(Main.java:1409) at org.eclipse.equinox.launcher.Main.main(Main.java:1385)
(In reply to comment #12) > It seems the performance results generation is failing with the following error > message. Is this related to this bug > > !ENTRY org.eclipse.test.performance 4 1 2010-11-17 09:18:51.016 > !MESSAGE Execution failed due to a distribution protocol error that caused > deallocation of the conversation. The command requested could not be completed > because of a permanent error condition detected at the target system. I guess the exception is because of this message. I am not able to connect to the server because of this error. I guess this is the reason for non availability of the I build results now.
(In reply to comment #11) > Created an attachment (id=183287) [details] > Sample of launch config to generate HTML results similarily as the Releng Ant > script does > > Satyam, feel free to contact me if you still have some troubles to reproduce or > debug this issue... Frederic, Thanks I am able to generate HTML results using this launcher and am not able to reproduce the problem. I am seeing the proper baselines as expected. Here are some other points to note. The N20101118-2000 generated results does show proper baseline (R-3.6-201006080911_201011121800). There was supposed to be a new baseline on 19th, but I don't see the results of them yet in the DB. It probably didn't run. The N20101120-2000 results are not generated( I don't know why), but I could see the results in the DB. These results are using the 12th baseline as the 19th is not available and doesn't seem to be OK too. While debugging, I could see that this could go wrong in two conditions: 1. Non-availability of the baseline build. 2. The particular test not having the result in the baseline build. I will monitor the tests in the coming weeks and get more details. By the way, the generated results should be correct unless the performance test is modified.
>By the way, the generated results should be correct unless the performance test >is modified. What about newly added ones?
(In reply to comment #15) > >By the way, the generated results should be correct unless the performance test > >is modified. > What about newly added ones? There's no problem for new added tests as soon as the same test has been correctly released in the performance reference stream (e.g. perf_36x for 3.7). BTW, I think I was too pessimistic on this issue. The baseline result is supposed to be flat for each performance test. Hence, it's not a real big issue not to take the latest one to generate performance results. If the baseline results is not enough stable, then the performance tools is able to notice it and we can remove the test from the verification, then avoid unnecessary performance alarm... So, I reduce the severity of this bug to normal and update the title to make it less frightening... :-S Note three things: 1) It seems to occur only on maintenance branch: for 3.7, it's always the last baseline which is used :-) 2) I was also not able to reproduce this issue locally :-( 3) I've updated the HTML pages with the generated results using last baseline...
Need some more time to validate this. As the severity is reduced, moving it to the next milestone.
Moving this again :(
Moving again :(
Still couldn't figure out the real reason. Kim thinks this could be because the baseline process still stays alive even after the test is completed - bug 345331 comment 9. Now, she made changes to kill the processes that have been hanging, this will probably get fixed - bug 343814 comment 8. Will have to wait and watch and hence moving it again.
Need more time to test and validate
(In reply to comment #21) > Need more time to test and validate Clicked the button too quick. Need more time to test and validate. Hence moving it to 3.8.
This bug hasn't had any activity in quite some time. Maybe the problem got resolved, was a duplicate of something else, or became less pressing for some reason - or maybe it's still relevant but just hasn't been looked at yet. If you have further information on the current state of the bug, please add it. The information can be, for example, that the problem still occurs, that you still want the feature, that more information is needed, or that the bug is (for whatever reason) no longer relevant. If the bug is still relevant, please remove the "stalebug" whiteboard tag.
