| Hi.
I understand the general wisdom of not denormalising information, as you clearly explain in your message, Achim. In my fucntional testing, I certainly suffered from the pain of having to keep my test scripts, maintained with an automated tool, and my related test cases and procedures, maintained with a text editor and/or a spreadsheet, in sync.
However I would like to share the reasons why I found the ancillary textual and tabular information invaluable: 1) Tracking: the automated test tools I used did not have any traceability maintaining and stats producing facilities; just staring at a large number of test script certaimly could not and did not give me an overall view of where the testing effort was at and what needed doing. In other words, a high level management perspective was totally lacking.
2) Monitoring of state of development: different test scripts were in different states in their life cycle (designed, implemented, in regression, etc.) and it would have been impossible for me to figure out which test script was in which state of development if I had not had some external control documents.
3) Coverage: this is really a subset of traceability, which I mentioned above. Since our development was (and still is) use case driven, all functional tests were based on use case flows and scenarios. Figuring out which use case was completely tested in which suite (especially in the case of included or extending use cases) would have been a nightmare without external matrices to specify it.
4) Bug identification: when a script failed in regression, I would go through the test case manually by repeating the steps as defined in the script. Having a textual document that clearly specified startup, steps and all conditions made the task of spotting the bug much much easier.
5) Test script reproducibility: If for any reason my test script got corrupted and I had to reproduce it, again having a detailed test case document to use as my guideline was pretty invaluable.
I understand when you talk about the different consumers of test cases. However I think that, even when an automatic test tool is chosen, as is our case, there are always circumstances in which human interaction or intervention is needed, as I have tried to highlight in my points above. This is why, at my first superficial reading of Testopia, I thought it might have quite a good role to play.
I just thought I would share my findings and my concerns. As I have said many times, my experience of functional testing is limited. Also it may well be that GUIDancer solves all issues I have mentioned above. I am looking forward to Achim's strategy document and his expertise to learn how the issues I have faced and tried to tackle in a simple, time consuming and perhaps naive, way, can be solved in a more automated, less error prone and more elegant fashion.
Thanks, B. On 19 Oct 2008, at 15:16, Achim Lörke wrote: Hi <dogma> You shall not have different sources for information! (Especially for something as time consuming as test descriptions and test results.) </dogma> From our experience one has to make a decision: use a tool to describe test cases for consumption by human testers or design automated tests for consumption by a test tool. If you try to do both you will end up with transferring requirements from one database to the other and results/modifications vice versa (just like UML diagrams and source code if not tightly integrated) . The two repositories will invariably drift apart. I have to admit that GUIdancer is not the best tool to describe test cases for humans (documentation support is planned for next year) but it’s doable. Therefore I wouldn’t introduce another tool for test tracking. Achim [Mat - please nix this up front if there is no chance, otherwise we will make an initial evaluation.] Yahya brought up the subject of Testopia last week at the team meeting but I have not had the chance to take a look at it until this evening. I for one misunderstood what is used for. It is not a testing tool, but a test tracking system from Mozilla. From the website: Testopia
Testopia is a test case management extension for Bugzilla. It is designed to be a generic tool for tracking test cases, allowing for testing organizations to integrate bug reporting with their test case run results. Though it is designed with software testing in mind, it can be used to track testing on virtually anything in the engineering process.
It looks as if it may be a useful addition to our testing toolkit, but the problem is that it requires patching Bugzilla. Achim/B. if you want to take a very quick look at it and if you come up with a positive verdict, I can ask the EF Web Masters to consider making it available. As that is a long shot, I would not spend a lot of time up front. ########################################### This message has been scanned by F-Secure Anti-Virus for Microsoft Exchange. For more information, connect to http://www.f-secure.com/ _______________________________________________ ormf-dev mailing list ormf-dev@xxxxxxxxxxxhttps://dev.eclipse.org/mailman/listinfo/ormf-dev
|
