Test Vector Optimization Based on Coverage
When coverage is the main test requirement, we can use testIDEA
command iTools | Optimize Test Vectors to select test cases,
which did not increase coverage. Then we can decide whether to delete
them or to move them under some other base test case, which is
executed less frequently. For example, suppose we have test cases
test-01, test-02, test-03, test-04, test-05,
and test-06 with the following coverage statistic values:
| Test case ID | Coverage |
|---|
| test-01 | 10% |
| test-02 | 23% |
| test-03 | 23% |
| test-04 | 53% |
| test-05 | 53% |
| test-06 | 100% |
We can see that tests test-03 and test-05 have the
same coverage statistic value as previous test, which means they
do not execute any previously not executed code. We can select all
such test cases with this command.
Usage
The recommended way to use this feature:
- Select a base test case, which has
derived test cases for the same function and with
coverage section defined. Do not select derived test cases!
- Select command Test | Run Selected And Derived to
execute test cases.
- After tests finish execution, and still with the base test
selected, execute command iTools | Optimize Test
Vectors.
- Decide what to do with the selected test cases.
Requirements
To optimize a set of test vectors, test cases must first be run with
coverage mode set to append. Only test cases, which have coverage
defined and at least one statistics section with at least one
statistics item criteria defined, can be considered for optimization
process. If any of the test cases does not fulfill the condition, an
error is issued.