Community
Participate
Working Groups
Users who create a datapool and variables cannot import from a csv file. Users should be able to import into a datapool from a csv once the datapool is created.
I'd like to understand the use case that motivates this request. Importing from CSV into a blank datapool is useful because some users have sets of data that are external to Hyades (that they have used for testing in the past), and they want to import that data. Importing into an existing datapool is a different question, because the data must match the existing datapool. (i.e. the columns in the CSV file must represent the same variables that already exist in the datapool.) This seems like a very fringe use case. I am cc'ing Bill and Dave, as they are the two primary consumers of the Hyades datapool editor, and they are very familiar with its many shortcomings. Bill & Dave, can you please comment on this request? Would this be a useful feature to have in the datapool editor for your users, and how would you prioritize it among the other datapool related defects / enhancement requests? Thanks, --Joe
Adding comments as a result of the Hyades UI Meeting on Sept 22, 2004: It is currently only possible to import a CSV to a new datapool during creation time. It is not possible to import a CSV file into an existing or empty datapool. An export Datapool/CSV feature exists, but there is not a matching import wizard. Round trip of import/export of datapools is missing. CCing David so he can elaborate (e.g., on the round trip issue, etc.)
An important scenario for datapool import/export is that population of large volumes of data in a datapool is an expensive operation. Typical users don't manually enter large number of entries since the time to generate thousands of entries manually is just prohibitive. Instead data in a datapool is often from an external source such as a database or utility that can programatically generate the necessary datapool values. But typically your test data doesn't drive or motivate your testing effort. Typically (in load testing scenarios for this example), the data values are generated by examining the parameters in the generated messages to the system under test. Then the values that are important and represent user provided input are typically replaced by datapool variables. This customization of a test to identify parameters which should be retrieved from a datapool does require analysis by the tester and the steps to map those parameters is non- trivial. This process creates the basic structure of the datapool in terms of the number of datapool variables. The initial record for the datapool then represents the original parameter values from the original generated test. This basic, simple datapool then provides the template by which the user decides how to generate a large amount of data which is similar but of the same type. So the use case is basically, to customize a test with random test data. Basic flow is: 1) Generated test with recorded parameter values. 2) User identifies parameters that are critical to randomize to generate load that is realistic model of actual user loads. 3) Simple datapool is created with appropriate number of datapool variables and initial record of datapool based on recorded parameter values 4) User exports datapool to allow import of data values from database or other utility 5) User imports datapool to existing datapool asset. This retains the datapool asset and the original datapool structure but provides additional records for the datapool. An important aspect in step 5 is that the existing datapool asset must be reused or the links from the test to the datapool variables is lost and must be remapped, at a minimum by pointing to the new datapool or worst relinking every datapool variable. Desire to minimize this remapping due to manual steps by user. Also creating new assets for the same "logical" item in the test generates problems if in a team situation and versioning is desired since version control system may treat the asset as a new file asset and comparisons between the two versions/files will generate extraneous deltas that aren't important in the perspective of the user who was just trying to generate additional data for the datapool. It is worth considering if instead of supporting import to an existing asset if this could be folded into a general mechanism of datapool test data generation which is the real underlying use case. Since some would consider importing to an existing asset atypical for the Eclipse import metaphor, maybe a better UI metaphor would be datapool generation. For example, instead of importing to an existing asset. user would invoke 'Generate Test Data' command from the editor and be able to select a CSV file (previously exported), and optionally a set of columns in that CSV file as the source of the data to generate. Since many interesting data generation schemes are also CSV-based generation from static data, we'd be very close to implementing another useful feature which is overall datapool generation at the same time. Then the right extension points to the 'Generate Test Data' command to support generation providers other than static CSV files such as integers between a certain range, strings of varying length, etc. could also be provided. Domain specific providers could be provided as well as other extensions to deal with other data types beyond string relevant to that domain (e.g. Point objects for a GUI test). This avenue leads us to enforce that structure of a datapool can't be changed when doing datapool generation since you could require the variable be defined as a requirement. The 'no-change-to-datapool-structure' is largely the same constraint that causes others to resist importing into an existing datapool since the number of variables/name of variable may change in the re-imported asset and dealing with the "merge conflicts" is what will make it difficult to implement and might lead the user to not understanding what limitations must be adhered to when reimporting the datapool. Food for thought, deserves some more analysis and discussion from the interested parties, let me know.
update based on requirements group review
checked in PII changes after initial feature design. Attached are mock-up screen shots for datapool import wizard and a warning message. There might be slight changes to the look after implementation. As for the initial design, importing to existing datapools will only be done either on a datapool with the same column/variable structure as the CSV file or an empty datapool. Also, when importing data to a datapool with existing records(rows), user will get prompted if they want to overwrite or append to it.
Created attachment 18992 [details] Disign of Datapool import wizard
[sizing = 1 week]
*** Bug 93108 has been marked as a duplicate of this bug. ***
Added support to import from CSV to an existing datapool. Importing is possible either to an empty datapool, or to a datapool that has the same variable information as the CSV file. New data fromm CSV will be appended to existing records(rows) in the datapool. In the new import wizard, number of variables will be checked between the selected CSV and datapool if the CSV does not contain variable info., or, variable name and type will be checked if the first line of the CSV contains variable info.
Currently there's a bug with datapol editor when a datapool is opened then new data is imported, Equivalence Class tab doesn't refresh its content. Close and reopen the datapool workaround it. This is tracked separatly by Bug 93383.
*** Bug 107133 has been marked as a duplicate of this bug. ***
ACTION: Please verify/close this defect.
closing
