Running tests

• Command Run All executes all tests defined in testIDEA.
• Command Run Selected executes all tests, which are selected in test tree. We can select multiple tests to run. Even if any of selected tests has derived tests, they are NOT run with this option.
• Command Run Selected And Derived executes all tests, which are selected in test tree. We can select multiple tests to run. If any of selected tests has derived tests, they are also run with this option.
• Command Run with Filter executes all tests, which satisfy conditions in filter. After executing this command we can select a filter to be used for tests specification execution. There exist two possibilities for fitering:
  • built-in filter - this filter is very flexible and should satisfy majority of use cases.
    All fields in tags section, which are empty, evaluate to true. This means, that with empty filter all tests will be executed. If we want to execute only some tests identified by function name and/or test ID, then we have to define a nonexisting tag in the Must have all tags field. For example, suppose we want to execute tests for functions min(), max(), and average():

    

    Note that we have used a tag, which is not used in any of our test specifications (in the example we used 'unusedTag'). Otherwise all tests would satisfy the tags filter. Now only tests specifications for functions mentioned as exceptions will be executed, while others will not pass the tags filter.
    The second part defines exceptions to the filtering with tags. All exceptions are evaluated in the same order as they are presented in the dialog. They override tag conditions.
    
  • script function - in this case a function in Python module is called. It is completely up to our implementation which test specifications will be executed. If the function returns true, test specification will be executed.

• Command Run Failed Tests executes all tests, which have failed in the previous run.
  If test suite contains a large number of tests, and only few of them fail, it may be time consuming to either select failed tests one by one or rerun all tests. In such cases we can fix the problem which has caused the tests to fail, and then use command Run | Run Failed Tests. Only the tests, which have failed on the previous run will be executed.

• Boolean menu option Keep test results toggles erasing of test results on test start. When the option is selected, the Run command does not erase results of tests, which are not executed. This way we can execute only tests, which have failed, and get report for all tests.

  IMPORTANT! Use this functionality with caution, because fix for one test may break other tests, which have already passed. Do not use this functionality on tests, which are interdependent, or which can be influenced by the fix of the problem. In such cases always rerun all tests!