| RE: [dsdp-mtj-dev] MTJ (Tests) |
|
Hi Edwin, Thanks for you valuable review. I’ve
made some rework based on your comments. Please can you verify them? I’ve also commented some of you questions
below. Questions left uncommented were corrected. Hugo From:
dsdp-mtj-dev-bounces@xxxxxxxxxxx [mailto:dsdp-mtj-dev-bounces@xxxxxxxxxxx] On Behalf Of Marinho Edwin-FMH678 My
considerations about the tests of the MTJ.
Is this
format of test case a pattern of eclipse? Could change to another format to
make it easier to be executed?
[h]There is no Eclipse standard. We’ve chosen this
format to cover as much as we can in these first cycles. After that we can try
to create some “procedural test cases” based on the results and
errors encountered. Do you have any other suggestion?
Preferences Pages
·
Walk through each of the
Preferences (All
preferences? Yes, all from MTJ ) and enable/disable (will only checkbox be available? Could
change “enable/disable” to “change the value”)
them (What about
the preferences of the item J2ME? There
is no J2ME item. J2ME was EclipseME naming, we changed to o
Check: The outcome is what you expect? o
Check: Are the Preferences documented
sufficiently? o
Check: The preferences can be restored to
default values? Device Management Page ·
Check: Can import
Devices ·
Check: Can edit
Devices ·
Check: Can delete Devices ·
Check: Can duplicate Devices New MIDlet Project (Should it be “MIDlet Suite”? The current naming convention used is MIDlet Project. “Suite”
removed where necessary) Page ·
Resource folder field (Does not cover all
functionalities We just
mentioned the main functionalities, but we hope testers exercise all of them as
tests are exploratory. The same applies for other tests below where you pointed
the same issue.) o
Check: Can specify a resource folder name o
Check: If you specify a folder name, the next project
you create uses it as resource folder? Over the Air Page ·
Check: Can enable preferences (which preferences?. Does it test the functionalities? changed) Packaging Page ·
Check: Can add/remove "Excluded Manifest Entries" (This verification does not
cover the functionalities of this page) Obfuscation Page ·
Check: Can configure Proguard settings Preverification Page ·
Check: "Use Built-in preverifier"
can't be selected (This
verification does not cover the functionalities of this page) Import Device
Wizard
·
Check: The Finish button is disabled before
finding a device? ·
Check: The Select All and Deselect All buttons
are disabled before finding a device? ·
Check: The Refresh button is disabled before
specifying the search directory? ·
Check: The Finish button is disabled before
finding a device? (Duplicated) removed ·
Check: Choosing an (a) folder without any
device inside and pressing Refresh
will result in no devices found (Will appear a dialog with this information? No, the list of devices found will remain empty) ·
Check: Choosing an folder with devices inside and
pressing Refresh
will result in the list of devices found (Could change to “will show the list
of devices the devices found” changed) ·
Check: Finishing the wizard (pressing Finish changed) after finding at least one device, will
add the device list to the Device
Management Page Device Editor
·
Basic
Page o
Check: fields can be edited and the changes reflect
in the Device
Management Page ·
Libraries
Page o
Check: can add and remove libraries o
Check: can define javadoc for library o
Check: can define source for library ·
Properties
Page o
Check: Cant (Can not) edit, add or
remove properties. MIDlet
Project
New MIDlet (Suite - see above) Project Wizard
·
Check: "Mobile
Tools for Java" category exists ·
"Mobile
Tools for Java" Category o
Check: MIDlet
(Suite – see above) (Project in category ·
MIDlet
Suite Properties o
Check: Can manage Devices o
Check: Can change Application descriptor name o
Check: When one device is selected and Application descriptor field is empty the Next and Finish buttons must
be disabled * o
Check: When one device is selected and Application descriptor field has an
invalid value (i.e. something.txt) the Next and Finish buttons must be disabled * o
Check: When no device is selected and Application descriptor field is empty the Next and Finish buttons must
be disabled * o
Check: When no device is selected and Application descriptor field has valid
value (i.e. something.jad) the Next and Finish buttons must be disabled * o
Check: When one device is selected and Application descriptor field has valid
value (i.e. something.jad) the Next and Finish buttons must be enabled (* Could group in only one
test)(grouped) ·
Java
settings o
Check: Can edit Java settings o
Libraries
Tab §
Check: "Java ME Library" corresponds
to the device selected in previous page. §
Check: If another device is selected, the
change is reflected in "Java ME Library" ·
Check: The wizard can be canceled at
any time. ·
Check: The user can go back to the previous
wizard page at any time ·
Check: The wizard can be finished at
any time. (What
about the case of not put the project name?) corrected Project
Properties
·
Check: "Mobile
Tools for Java" category exists ·
Check: Can change project device (This change
must reflect on project's "Java ME Library") ·
Check: Can change JAD file name ·
Check: Can change JAR file name ·
MIDlet
Suite Signing o
Check: Can configure signing settings o
Check: If correctly configured the MIDlet project must be correctly signed ·
Obfuscation
o
Check: If "Enable project specific
settings" is enabled, this (these?) settings must
override the default configuration o
Check: Can configure obfuscation settings ·
Packaging
o
Check: If "Enable project specific
settings" is enabled, this (these?) settings must
override the default configuration o
Check: Can configure packaging settings ·
Preverification
o
Check: If "Enable project specific
settings" is enabled, this (these?) settings must
override the default configuration o
Check: Can configure preverification settings Jad
Editor
·
Check: Any property edited by the Jad Editor must be updated in the <project_jad_file>.jad o
Required
Properties Page o
MIDlets
Page o
Optional
Properties Page o
Over
the Air Properties Page o
Push
Registry Page o
Used
Defined Properties Page Project's Context Sensitive Menu
·
MIDlet
Suite Project o
Check: Can create package o
Check: If "Proguard" is correctly
configured, can create obfuscated package o
Check: If "Antenna" is correctly
configured, can export Antenna build files. ·
Regular
Java Project/Other Project o
Check: Can convert to MIDlet Project Run/Debug
Dialog
·
MIDlet
tab o
Check: Can edit settings ·
Emulation
tab (Does
not cover all functionalities) o
Check: Can set a specific device to run/debug o
Check: If "Specific Device" is
selected, can manage devices (The
tests and the requirements should have all functionalities covered. On the
requirements have functionalities that are not tested. On the test cases have
functionalities being covered that are not documented on the requirements.) The
following requirements were not covered by tests: Workflow: tests added FR002, FR007, FR008, FR009 MTJ Build: FR001 covered in context sensitive menu test (can
create package), FR002 this requirement is not valid anymore and
will be removed, FR005 covered in context sensitive menu test (can
create package), FR006 covered in context sensitive menu test (can
create obfuscated package), Device Management: Does not cover the
available scenarios –
covered by different test environments (varying
SDK) Non Functional: NFR001, NFR002, NFR003,
NFR004 – covered by
different test environments (varying: OS, Eclipse version, SDK) NFR005, NFR006, NFR007,
NFR008 – don’t
know how to test this, as this libraries are shipped with MTJ. Any suggestions Edwin Carlo R. Marinho |