platform-core-home/documents/test_plan_30.html

Parent Directory Parent Directory | Revision Log Revision Log


Revision 1.16 - (view) (download) (as text)

1 : dj 1.1 <!doctype html public "-//w3c//dtd html 4.0 transitional//en">
2 :     <html>
3 :     <head>
4 :     <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
5 :     <meta name="GENERATOR" content="Microsoft FrontPage 4.0">
6 :     <title>Test Plan - 3.0</title>
7 :     <link rel="stylesheet" href="http://dev.eclipse.org/default_style.css" type="text/css">
8 :     </head>
9 :     <body text="#000000" bgcolor="#FFFFFF">
10 : jeff 1.10
11 : jeff 1.5 <h1><a name="top"></a>3.0 Core Test Plan</h1>
12 : jeff 1.7 <p>Testing will cover the functional areas on these platforms/VMs</p>
13 :     <table border="1" width="63%">
14 : jeff 1.5 <tr>
15 : jeff 1.7 <td width="54%">&nbsp;</td>
16 :     <td width="23%">JDK 1.4.2</td>
17 :     <td width="23%">SC 1.4.2</td>
18 : jeff 1.5 </tr>
19 :     <tr>
20 : jeff 1.7 <td>Windows 2000 (FAT32)</td>
21 : jeff 1.5 <td>&nbsp;</td>
22 :     <td>&nbsp;</td>
23 :     </tr>
24 :     <tr>
25 : jeff 1.7 <td>Windows XP (NTFS)</td>
26 : jeff 1.5 <td>&nbsp;</td>
27 :     <td>&nbsp;</td>
28 :     </tr>
29 :     <tr>
30 :     <td>Mac OSX</td>
31 :     <td>&nbsp;</td>
32 :     <td>N/A</td>
33 :     </tr>
34 :     <tr>
35 :     <td>Linux</td>
36 :     <td>&nbsp;</td>
37 :     <td>&nbsp;</td>
38 :     </tr>
39 :     </table>
40 :     <ul>
41 :     <li><a href="#launcher">Launcher</a></li>
42 :     <li><a href="#osgi">OSGi</a></li>
43 : dj 1.16 <li><a href="#equinox">Runtime - Equinox</a></li>
44 :     <li><a href="#jobs">Runtime - Jobs</a></li>
45 :     <li><a href="#preferences">Runtime - Preferences</a></li>
46 : jeff 1.5 <li><a href="#resources">Resources</a></li>
47 :     <li><a href="#pde-build">PDE Build</a></li>
48 :     <li><a href="#performance">Performance</a></li>
49 :     </ul>
50 : jeff 1.6 <p>Points to remember when testing</p>
51 :     <ul>
52 :     <li>File a bug when something does not work. Do not try to fix the problem
53 :     unless it prohibits further testing.</li>
54 :     <li>When
55 :     a test can be automated and is not part of our test suite, add a JUnit
56 :     test to the test suite.</li>
57 :     </ul>
58 : jeff 1.5 <hr>
59 : dj 1.1
60 : johna 1.12
61 : dj 1.1 <h3><a name="launcher"/>Launcher</h3>
62 : jeff 1.3 <h4>No config.ini</h4>
63 : dj 1.1 <ul>
64 : jeff 1.3 <li>install Eclipse</li>
65 :     <li>delete the configuration dir</li>
66 :     <li>start Eclipse</li>
67 :     <li>confirm that everything goes smoothly. The only thing that should be different
68 :     is the lack of splash screen.</li>
69 :     <li>Note: There may be an issue with the eclipse.product not being set and the
70 :     runtime not knowing which application to run. We may no longer be able
71 :     to run without a config.ini. If there is a strong usecase for this then
72 :     we could embed a valud in eclipse.properties if we had to.</li>
73 : dj 1.1 </ul>
74 : jeff 1.7 <h4>Exit data</h4>
75 :     <ul>
76 : jeff 1.8 <li>set the exit data system property to a string value</li>
77 :     <li>exit Eclipse with the an error code other than 0, 23 or 24 and ensure
78 : jeff 1.7 the exit data string is properly displayed</li>
79 : jeff 1.8 </ul>
80 : jeff 1.7 <h4>Restart</h4>
81 :     <ul>
82 :     <li>start Eclipse, use the VM, commands and vmargs System properties to construct
83 :     a new command line</li>
84 :     <li>set this new command line in the exitdata</li>
85 :     <li>exit Eclipse with return code 24</li>
86 :     <li>confirm that the specified command was run on restart</li>
87 :     </ul>
88 :     <a href="#top">Back to top</a><br><hr>
89 : dj 1.1
90 :     <h3><a name="osgi"/>OSGi</h3>
91 : jeff 1.6 <h4>Command line options</h4>
92 :     <ul>
93 : jeff 1.8 <li>try various command line options and ensure that the corresponding system
94 : jeff 1.6 property values are set. In particular,
95 :     <ul>
96 :     <li>-configuration</li>
97 :     <li>-product</li>
98 :     <li>-application</li>
99 :     <li>-install</li>
100 :     <li>-data</li>
101 :     </ul>
102 :     </li>
103 :     </ul>
104 : jeff 1.2 <h4>Shared Install 1</h4>
105 : dj 1.1 <ul>
106 : jeff 1.2 <li>install Eclipse on shared drive</li>
107 :     <li>run once with -initialize, mark read-only</li>
108 :     <li>run one instance, exit</li>
109 :     <li>run another instance (as different user or on different machine), exit</li>
110 :     <li>run two instances at the same time (as different users or on different
111 :     machines)</li>
112 :     <li>exit in same order as start (i.e., start 1, start 2, exit 1, exit 2)</li>
113 :     </ul>
114 :     <h4>Shared Install 2</h4>
115 :     <ul>
116 :     <li>install Eclipse on shared drive</li>
117 :     <li>run once with -initialize, mark read-only</li>
118 :     <li>run user instance, exit</li>
119 :     <li>start admin instance (with read/write permissions)</li>
120 :     <li>install new features using update manager, exit</li>
121 :     <li>restart previous user instance</li>
122 :     <li>confirm that newly installed features are present and funcitonal</li>
123 :     <li>check that no additional files have been written in the local configuration
124 :     area</li>
125 :     </ul>
126 :     <h4>Shared Install 3</h4>
127 :     <ul>
128 :     <li>install Eclipse on shared drive</li>
129 :     <li>run once with -initialize, mark read-only</li>
130 :     <li>run one instance</li>
131 :     <li>install new features using update manager</li>
132 :     <li>restart. New features should be there and totally functional</li>
133 :     <li>start another instance as the same user on the same machine</li>
134 : jeff 1.8 <li>previously installed features should be present and functional</li>
135 : jeff 1.2 <li>start another instance on a different machine or as a different user</li>
136 :     <li>previoulsy installed features should NOT be present </li>
137 : dj 1.1 </ul>
138 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
139 : dj 1.1 <h3><a name="equinox"/>Runtime - Equinox</h3>
140 :    
141 : jeff 1.3 <h4>JAR'd plugins</h4>
142 : dj 1.1 <ul>
143 : jeff 1.3 <li>populate an update site with the Core Tools feature and plugins/fragments.
144 :     mark the relevant feature entries as unpack=false.</li>
145 :     <li>start Eclipse and install the Core Tools feature using update manager</li>
146 :     <li>check that the plugins and fragments did not get exploded</li>
147 :     <li>confirm that the Core Tools function as expected</li>
148 : dj 1.1 </ul>
149 : rchaves 1.9 <h4>Extension registry</h4>
150 :     <ul>
151 :     <li>install a plug-in dynamically and check whether its extensions and extension
152 :     points are properly added to the registry</li>
153 :     <li>remove it and ensure its extensions and extension points go away</li>
154 :     <li>do the same as above with two plugins where one of them requires the other.</li>
155 :     <li>do the same as above with a plugin and a fragment, where both contribute extensions/extension points.</li>
156 :     <li>ensure plugin manifests with valid XML but missing required elements/attributes
157 :     in extensions/extension points markup are properly ignored/handled</li>
158 : jeff 1.10
159 : rchaves 1.9 </ul>
160 : rchaves 1.15 <p><strong>Note</strong>: use the <code>org.eclipse.core.runtime/registry/debug/events/extension</code> debug option
161 :     to monitor registry change events</p>
162 : jeff 1.10 <h4>Nested jars</h4>
163 :     <ul>
164 :     <li>Zip the Junit folder such that the files are at the root of the archive, and rename the archive to have .jar as extension</li>
165 :     <li>Delete the JUnit folder</li>
166 :     <li>Start eclipse</li>
167 :     <li>In the console type
168 :     <pre>
169 :     install reference:file:/<junit.jar>
170 :     refresh <installNumber of the installed plugin>
171 :     ss
172 :     </pre>
173 :     Check that the junit plugin is installed and resolved.
174 :     Run a Junit test.
175 :     </ul>
176 :    
177 :     <h4>Code in folders</h4>
178 :     <ul>
179 :     <li>Unzip the content of the original JUnit.jar into a folder named "code"</li>
180 :     <li>In the plugin.xml replace &gt;library name="junit.jar"&lt; by &gt;library name="code/"&lt;</li>
181 :     <li>Start eclipse</li>
182 :     <li>In the console type
183 :     <pre>
184 :     install reference:file:/<junit folder>
185 :     refresh <installNumber of the installed plugin>
186 :     ss
187 :     </pre>
188 :     Check that the junit plugin is installed and resolved.
189 :     Run a Junit test.
190 :     </ul>
191 :    
192 :    
193 :     <h4>Folders in a Jar</h4>
194 :     <ul>
195 :     <li>Zip the content of the Junit folder from the previous test in such a way that plugin.xml is that the root of the archive</li>
196 :     <li>Start eclipse</li>
197 :     <li>In the console type
198 :     <pre>
199 :     install reference:file:/<junit folder>
200 :     refresh <installNumber of the installed plugin>
201 :     ss
202 :     </pre>
203 :     Check that the junit plugin is installed and resolved.
204 :     Run a Junit test.
205 :     </ul>
206 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
207 : dj 1.1 <h3><a name="jobs"/>Runtime - Jobs</h3>
208 : johna 1.12 <h4>Test concurrent operations in UI thread</h4>
209 :     <ul>
210 :     <li>Enable <b>Always run in background</b> preference </li>
211 :     <li>Start checkout of platform-core module from CVS</li>
212 :     <li>During checkout, switch to the Java perspective</li>
213 :     <li>UI may become blocked, but should eventually become responsive</li>
214 :     <li>Open Path.java in Java editor</li>
215 :     <li>Switch back to CVS perspective</li>
216 : johna 1.13 <li>Start checkout of org.eclipse.jdt.ui.tests.refactoring (or other big project)</li>
217 : johna 1.12 <li>Open Navigator: New files should start appearing</li>
218 :     <li>Add breakpoint to a method in Path.java: should appear quickly</li>
219 :     <li>Remove breakpoint: should be responsive</li>
220 :     <li>Start typing in Path.java: Should not block</li>
221 :     <li>Save Path.java: Should not block</li>
222 :     <li>Start typing in Path.java again: Should not block</li>
223 :     <li>Revert file: Should not block</li>
224 :     </ul>
225 :     <h4>Many background tasks</h4>
226 :     <ul>
227 :     <li>Start CVS checkout of ten or more different projects</li>
228 :     <li>Open progress view (double-click in lower right hand corner of workbench window)</li>
229 :     <li>All checkout tasks should be making progress</li>
230 :     <li>Canceling the checkouts should respond in a timely manner.</li>
231 :     </ul>
232 :     <h4>Nested blocking acquires</h4>
233 :     <ul>
234 :     <li>Install the org.eclipse.ui.examples.job plugin</li>
235 :     <li>Load org.eclipse.ui.examples.job</li>
236 :     <li>Open the Job Factory view</li>
237 :     <li>Check "Lock the workspace"</li>
238 :     <li>Click "Create Jobs"</li>
239 :     <li>Change quantity to 3 and duration to 10 seconds</li>
240 :     <li>Click "Touch Workspace"</li>
241 :     <li>Cancel the Test Job</li>
242 :     <li>Dialog should close</li>
243 :     <li>Repeat all above steps except cancel the blocked operation instead of the test job (three times)</li>
244 :     <li>Dialog should close but test job should continue running</li>
245 :     </ul>
246 :     <h4>Shutdown while job is running</h4>
247 :     <ul>
248 :     <li>Start a long CVS checkout</li>
249 :     <li><b>File &gt; Exit</b>.</li>
250 :     <li>A progress dialog will appear indicating that the operation is waiting.</li>
251 :     <li>Expand <b>Details</b> area and cancel the CVS checkout</li>
252 :     <li>Exit should now complete in a timely manner</li>
253 :     </ul>
254 : dj 1.1 <h4>test1</h4>
255 :     <ul>
256 :     <li>item 1</li>
257 :     </ul>
258 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
259 : dj 1.1 <h3><a name="preferences"/>Runtime - Preferences</h3>
260 :    
261 :     <h4>test1</h4>
262 :     <ul>
263 :     <li>item 1</li>
264 :     </ul>
265 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
266 :     <h3><a name="resources"/>Resources</h3>
267 : dj 1.1
268 :     <h4>test1</h4>
269 :     <ul>
270 :     <li>item 1</li>
271 :     </ul>
272 : rchaves 1.9
273 :     <h4>Encoding - Content types</h4>
274 :     <ul>
275 :     <li>confirm that the right content type is selected when a complex hierarchy
276 : rchaves 1.14 of content types exist (use the resource properties dialog to see the content
277 :     type that was chosen):
278 :     <ul>
279 :     <li>a .xml file is recognized as XML</li>
280 :     <li>a .xml file whose root element is &lt;project&gt; is recognized as an
281 : rchaves 1.9 Ant Build Script</li>
282 : rchaves 1.14 <li>a plugin.xml file is recognized as a Plugin Manifest file</li>
283 :     <li>a plugin.xml file whose root element is &lt;project&gt; is recognized
284 : rchaves 1.9 as an Ant Build Script</li>
285 : rchaves 1.14 </ul>
286 : rchaves 1.9 </li>
287 : rchaves 1.14 <li>try the same as above with partial/empty XML files - whenever a file has
288 :     a .xml extension, it should at least be recognized as XML. If the contents
289 :     are complete enough (root element can be parsed), more specific content types
290 :     (such as Ant scripts) should be picked.</li>
291 : rchaves 1.9 <li>confirm that the right encoding is picked for XML files containing a &quot;encoding&quot;
292 : rchaves 1.14 attribute or not (default is UTF-8)</li>
293 :     <li>this is a list of all content types in the SDK:
294 :     <ul>
295 :     <li>org.eclipse.core.runtime.text
296 :     <ul>
297 :     <li>org.eclipse.core.runtime.xml (*.xml, .classpath, .properties)
298 :     <ul>
299 :     <li>org.eclipse.ant.core.antBuildFile (*.xml with a &lt;project&gt;
300 :     root element)</li>
301 :     <li>org.eclipse.pde.core.pluginManifest (plugin.xml)</li>
302 :     <li>org.eclipse.pde.core.fragmentManifest (fragment.xml)</li>
303 :     </ul>
304 :     </li>
305 :     <li>org.eclipse.jdt.core.javaProperties (*.properties)
306 :     <ul>
307 :     <li>org.eclipse.pde.core.pluginProperties (plugin.properties)</li>
308 :     </ul>
309 :     </li>
310 :     <li>org.eclipse.jdt.core.JARManifest (MANIFEST.MF)</li>
311 :     <li>org.eclipse.jdt.core.javaSource</li>
312 :     </ul>
313 :     </li>
314 :     </ul>
315 :     </li>
316 :     </ul>
317 :     <h4>Encoding - BOMs</h4>
318 :     <ul>
319 :     <li>confirm text files (or any derived content types such as XML and Java source
320 :     files) are said to have the right encoding when they present a Byte Order
321 :     Mark (use Notepad to generate those)</li>
322 :     <li>confirm opening such files works ok</li>
323 :     <li>confirm encoding of files with BOM can be overridden by setting a user-specified
324 :     encoding</li>
325 : rchaves 1.9 </ul>
326 :     <h4>Encoding - Preferences</h4>
327 :     <ul>
328 :     <li>check the encoding-related info is properly updated when changes are made
329 :     directly to the underlying preferences file (&lt;project&gt;/.settings/org.eclipse.core.resources.prefs)
330 :     - try editing the file in a text editor, deleting the file, making the file
331 :     out-of-sync</li>
332 :     </ul>
333 : dj 1.1
334 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
335 : dj 1.1 <h3><a name="pde-build"/>PDE-Build</h3>
336 : jeff 1.3 <h4>Plugin with "." on the classpath</h4>
337 :     <ul>
338 :     <li> Using the "new plugin wizard" create a plugin named Dot1 with "." as a jar name. </li>
339 :     <li> Export it as a zip. Verify that the classes are not into a jar. </li>
340 :     <li> Export it as folder. Verify that the classes are at the root of the folder.</li>
341 :     </ul>
342 :    
343 :    
344 :     <h4>Compiling against a plugin with "." on the classpath</h4>
345 :     <ul>
346 :     <li> Create a plugin named Normal</li>
347 :     <li> add a dependency to Dot1 in the plugin.xml</li>
348 :     <li> add a code dependency from a class in Normal to a class in Dot1</li>
349 :     <li> try to export Normal</li>
350 :     <li> try to export Normal and Dot1</li>
351 :     <li> confirm that no compile errors occured (if there were you would get an error dialog at the end of the export)</li>
352 :     </ul>
353 :    
354 :    
355 :     <h4>Plugin with a folder on the classpath</h4>
356 :     <ul>
357 :     <li> Create a plugin named Folder1 with "myFolder" as a jar name</li>
358 :     <li> edit the build.properties and add a slash after myFolder in bin.includes property</li>
359 :     <li> Export as folder, verify that the class files are rooted in a folder called myFolder</li>
360 :     </ul>
361 :    
362 :    
363 :     <h4>Testing the build order</h4>
364 :     <ul>
365 :     <li> in the plugin Folder1, add the following things in the runtime section of the plugin.xml</li>
366 :     <pre>
367 :     &lt;library name="yourFolder"&gt;
368 :     &lt;export name="*"/&gt;
369 :     &lt;/library&gt;
370 :     &lt;library name="library.jar"&gt;
371 :     export name="*"/&gt;
372 :     &lt;/library&gt;
373 :     </pre>
374 :     <li> replace the content of the build.properties by </li>
375 :     <pre>
376 :     source.myFolder = src/
377 :     output.myFolder = bin/
378 :     source.yourFolder = src/
379 :     output.yourFolder = bin/
380 :     source.library.jar = src/
381 :     output.library.jar = bin/
382 :     bin.includes = plugin.xml,\
383 :     myFolder/,
384 :     yourFolder/
385 :     jars.compile.order = yourFolder, library.jar, myFolder
386 :     </pre>
387 :     <li> generate manually the build.xml and verify that the build.jars target successively and in this order list yourFolder, library.jar and myFolder</li>
388 :     </ul>
389 :    
390 :     <h4>Reuse the plugin Normal</h4>
391 :     <ul>
392 :     <li> add a dependency to Folder1</li>
393 :     <li> add a code dependency from a class in Normal to a class in Folder1</li>
394 :     <li> Export Normal</li>
395 :     <li> Export Normal and Folder1</li>
396 :     </ul>
397 :    
398 :    
399 :     <h4>Export of qualified plugins</h4>
400 :     <ul>
401 :     <li> Create a plugin Qualified where the version number ends is 1.0.0.qualifier.</li>
402 :     <li> open the build.properties and add
403 :     <pre>qualifier = context</pre></li>
404 :     <li> Export the plugin as zip and verify that the name of the zip has 1.0.0.<date Of The Day></li>
405 :     <li> Export the plugin as folder and verify that the folder as the name 1.0.0.<date of the day></li>
406 :     <li> Verify that the version number of the feature in the feature.xml has been updated</li>
407 :    
408 :     <li> change the build.properties to
409 :     <pre>qualifier = NONE</pre></li>
410 :     <li> Export the plugin as zip and verify that the name of the zip is 1.0.0</li>
411 :     <li> Export the plugin as folder and verify that the name of the directory is 1.0.0</li>
412 :     <li> change the build.properties to
413 :     <pre>qualifier = myValue</pre>
414 :     </li>
415 :     <li> Export the plugin as zip and verify that the name of the zip is 1.0.0.myValue</li>
416 :     <li> Export the plugin as folder and verify that the name of the directory is 1.0.0.myValue</li>
417 :     </ul>
418 :    
419 :    
420 :     <h4>Export of qualified features</h4>
421 :     <ul>
422 :     <li> Create a feature QualifiedFeature, set the version number as 1.0.0.qualifier (don't need to add a plugin in the feature)</li>
423 :     <li> repeat the same steps as in the test with plugin</li>
424 :     </ul>
425 :    
426 :    
427 :     <h4>Export of qualified plugins and features</h4>
428 :     <ul>
429 :     <li> Take the feature previously created</li>
430 :     <li> add the plugin "Qualified" in the feature</li>
431 :     <li> Export the feature and verify that the feature.xml content has been updated (the reference to Qualified has the number from the "Qualified" plugin)</li>
432 :     <li> Verify that the version number of the feature in the feature.xml has been updated</li>
433 :     </ul>
434 :    
435 :    
436 :     <h4>Export of nested features with qualifiers</h4>
437 :     <ul>
438 :     <li> create a new feature (no importance in the name and version)</li>
439 :     <li> add the newly created feature to this feature</li>
440 :     <li> export the "QualifiedFeature" feature and verify that the feature.xml has been updated properly</li>
441 :     </ul>
442 : dj 1.1
443 : prapicau 1.11 <h4>Test of optional matching rules</h4>
444 :     <ul>
445 :     <li>Create a plugin named Match1</li>
446 :     <li>Create a plugin named Match2</li>
447 :     <li>Update Match2 dependencies to optionaly requires Match1</li>
448 :     <li>Close Match1 project</li>
449 :     <li>Generate the build.xml (right click on plugin.xml)</li>
450 :     <li>If the build.xml is generated it is ok</li>
451 :     </ul>
452 :    
453 :    
454 :     <h4>Tests of matching rules</h4>
455 :     <ul>
456 :     <li>Re-open the plugin named Match1</li>
457 :     <li>Update Match2 dependencies to have a dependency on Match1 which is version="1.0.0" match="perfect"</li>
458 :     <li>Generate the build.xml (right click on plugin.xml)</li>
459 :     <li>If the build.xml is generated it is ok</li>
460 :    
461 :     <li>Repeat steps 2, 3 and 4 for the 3 different values for matching rules
462 :     <pre>
463 :     version="1.0.0" match="perfect"
464 :     version="1.0.0" match="equivalent"
465 :     version="1.0.0" match="compatible"
466 :     version="1.0.0" match="greaterOrEquals"
467 :     </pre>
468 :    
469 :     <li>Change Match1 version number to be 1.0.1 and repeat the "repeat section". The test with perfect match is expected to
470 :    
471 :     fail.</li>
472 :     <li>Change Match1 version number to be 1.1.0 and repeat the "repeat section". The tests with perfect and equivalent are
473 :    
474 :     expected to fail.</li>
475 :     <li>Change Match1 version number to be 2.0.0 and repeat the "repeat section". The tests with perfect, equivalent and
476 :    
477 :     compatible are expected to fail.</li>
478 :     </ul>
479 :    
480 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
481 : dj 1.1 <h3><a name="performance"/>Performance</h3>
482 : jeff 1.8 <h4>Startup</h4>
483 : dj 1.1 <ul>
484 : jeff 1.8 <li>follow the empty, medium and big workspace scenarios as detailed on <a href="http://eclipse.org/eclipse/development/performance/index.html">Eclipse
485 :     performance page</a>.</li>
486 : dj 1.1 </ul>
487 : jeff 1.5 <a href="#top">Back to top</a><br><hr>
488 : dj 1.1
489 :     </body>
490 :     </html>