Writing Script Extensions

Script Extension Points

• Test case extensions
• Test execution extensions
• GUI extensions

Note: Path to Python interpreter is the same as used by winIDEA. If you need to change it, please see winIDEA dialog Tools | Options, tab Scripts.

Test case extensions

There exist six extension points in test case:

• Init target script function, section Scripts.
  Called before the test starts. No modifications of target stack are made up to this point, so we may run/stop the target if necessary. Test local variables declared in the Variables section of test case are _not_ available at this point, and target global variables are not modified by testIDEA. It is not recommended to set any variables in this function, as they might be overwritten by testIDEA when setting up test case. Use Init test script function for this purpose. If you set target global variables here, make sure they are not set in test case specification.
• Init test script function, section Scripts
  Called after the test is initialized, but just before the function under test is executed on the target. The stack on the target is set up at this point, and test local variables are created and initialized according to test specification. Target global and test persistent variables are initialized. If we need to initialize some large array, for example, this is the right point to do it.
  Function parameters can be accessed as variables with names defined as:

  ic.CTestCaseController.POSITION_PARAM_NAME_PREFIX + index

  where index is 0 for the first parameter, 1 for the second, ... Example:

    
          self.param0 = ic.CTestCaseController.POSITION_PARAM_NAME_PREFIX + '0'
          self.testCtrl.modify(self.param0, "1234")
          

  If parameter 0 is char array, we can write:

    
          self.testCtrl.modify(self.param0 + '[1]', "'a'")
          

  The target should not be started at this point, so calls like run(), stepOver(), call()... from CExecutionController are not allowed here.
• End test script function, section Scripts
  Called immediately after the function under test returns. The target state is not modified, so we can access global variables and test local variables on the stack. If we need to verify contents of arrays modified by function under test, for example, we should do it at this point.
  The target should not be started at this point, so calls like run(), stepOver(), call()... from CExecutionController are not allowed in this script function.
• Restore target script function, section Scripts
  Called after the test completes. The stack is cleared at this point, global variables are still available. Usually we do some final cleanup at this point, for example restore global variables.
• Stub script function, section Stubs
  Called when the stub is executed. This function is called whenever the function, which we have stubbed is called. This script function can be called more than once during one test case run. Parameters of the stubbed function are available at this point.
• Test Point script function, section Test points
  Called when test point is hit. This script function can be called more than once during one test case run. Function local variables are available at this point - script function can test and modify them.

Writing the script extension

Script functions should be implemented as methods of class in Python module. First we create a file with extension py. File name is also the name of the module. This name should be specified in the Imported modules field in the configuration dialog above. This way testIDEA can tell Python interpreter where to find our code.

Next we write a class in the created file. It is a normal Python class with methods. Names of the methods must be the same as given in test case. The number of parameters must also match.

Example of class with one extension method:

      class SampleExtension:

          def __init__(self, mccMgr = None):
              pass

          def initTarget(self):
              print 'Executing initTarget()'
    

    import isystem.connect as ic

    class SampleExtension:

        def __init__(self, mccMgr = None):
            """
            Normally we'll connect to winIDEA, which is running the test, so that
            target state can be accessed/modified by this script.
            """
            if mccMgr == None:
                # Executed when called from testIDEA.
                # Connection can't be passed between processes.
                self.mccMgr = None
                self.connectionMgr = ic.ConnectionMgr()
                self.connectionMgr.connectMRU()
                self.debug = ic.CDebugFacade(self.connectionMgr)
            else:
                # Executed when called from generated script - connection is reused.
                self.mccMgr = mccMgr
                self.connectionMgr = mccMgr.getConnectionMgr('')
                self.debug = mccMgr.getCDebugFacade('')

            self.testCtrl = None

    
        def initTarget(self):
            print 'Executing initTarget()'
            for i in  range(0, 10):
                self.debug.modify(ic.IConnectDebug.fMonitor, 'g_charArray1[' + str(i) + ']',
                                  str(i*i))
    

How can extension methods provide feedback?

• Throwing an exception indicates a failure of test execution. The test in testIDEA ends with exception in such case, and should be considered as not executed. The problem in this case lies in test case itself, not in the code on target.
• Text printed to std out is shown in console when running tests from console, and in Status view, when tests are started from testIDEA. This text is not shown in reports, it is intended to be used during test development for debugging purposes only.
• Function return values are the most important piece of information returned by extension functions. If function returns None, it means no error and test execution continues normally. If the function returns non-empty string, it is considered as error in test results, and the string contents is shown in test reports as the reason of test failure. Make sure the function returns only None or string, but no other types.
• Reserved variables can be used to provide additional information, which does not mean an error. For example, value of analog voltage should be in range for the test to pass, but we would like to have the actual value measured in the report. For such data each type of extension function has its own variable. Names of these variables are composed of prefix _isys_, followed by tag used in test specification for extension function, and followed by postfix Info. All extension functions and corresponding variables are shown in the following table:

  Extension (tag)	Variable name
  Initialize Target (initTarget)	_isys_initTargetInfo
  Initialize Variables (initFunc)	_isys_initFuncInfo
  Verify Variables (endFunc)	_isys_endFuncInfo
  Restore Target (restoreTarget)	_isys_restoreTargetInfo
  Stubs (stub: script)	_isys_stubInfo
  Test Points (testPoint: script)	_isys_testPointInfo

  Example of setting a variable in Python script:

            self._isys_initTargetInfo = 'information from initTarget()'
        

  Variables are set to None by testIDEA after extension function returns.

Return values and info variables of each function can be seen in tool-tips of decorations next to extension function name in section Scripts in testIDEA. Red decoration means error, green one means information is available, while no decoration means no error and no information:

How to access target variables

• global variables of the target application
• test local variables defined in test case section Variables
• parameters of target function (tested or stubbed)
• return values of target function (tested or stubbed)
• local variables of the target functions (tested or stubbed)

    def myExtFunction(self, testSpec):
        if self.testCtrl == None:
            self.testCtrl = ic.CTestCaseController(self.connectionMgr,
                                                   self._isys_testCaseHandle)
	

        # 'nItems' is the name of parameter specified in 'Parameters' field of the
        # 'Stubs' section in testIDEA. 'srv' is the name of return value as
        # specified in the same section.
        # Format specifier 'd' in 'nItems,d' ensures that the returned string 
        # always contains decimal value.
          
        nItems_t = int(self.testCtrl.evaluate('nItems,d'))
        print 'nItems = ', nItems_t

        self.testCtrl.modify('srv', str(nItems_t * 2))
        

How to access test specification

Reserved identifiers

Test execution extensions

• Test case filtering
  The function for test filtering is specified in filter settings (command Run | Run with filter). It should return None if the test should NOT be executed, and any non-empty string when the test should be executed. Currently the string returned is not used, but in the future it may be stored in test reports, so the recommended value is description why the test was selected for execution, for example: This function is part of communication module.
• Initialization sequence
  Initialization sequence is defined in File | Properties | Initialization sequence. It can contain several calls to script functions.
• Before saving test report
  Function isys_getTestReportCustomData(self, reportConfig) can be used to provide data to be used in test report. When implemented, it is called when user selects command Test | Save test report, before the dialog opens. The function should set variable self._isys_customScriptMethodInfo to string containing key: value pairs, one pair per line. These data is then added to report Test environment. Examples of usage are revisions of tested components.
  Example:
  

              def isys_getTestReportCustomData(self, reportConfig):
              
                   # get revisions as mapping
                   data = {'_appRev': 12355, 'bootloaderRev': '1.05g'}
  
                   # create string of 'key: value' pairs suitable for test report
                   dataAsList = [(k + ': ' + str(v)) for k, v in data.items()]
                   self._isys_customScriptMethodInfo = '\n'.join(dataAsList)
            

  If key starts with an underscore, it is saved to test report, but it is not saved to iyaml file. Other key/value pairs are saved also to iyaml file.

  

  Return value should be None if no error occurred, error message as string otherwise.
  This function has a predefined name - there is no setting in testIDEA.
• After test report is saved
  Function isys_afterReportSave(self, reportConfig) can be used to post-process test report, or send/commit it to a database, for example.
  When implemented, it is called after test report is saved. Return value should be None if no error occurred, error message as string otherwise.
  This function has a predefined name - there is no setting in testIDEA.

    testIDEAVersion: 9.12.279
    winIDEAVersion: 9.12.279
    reportContents: full
    outFormat: xml
    fileName: 'report\reportFull.xml'
    iyamlFileName: d:\bb\trunk\sdk\targetProjects\testIDEADemo.iyaml
    xsltFull: ' isystemTestReport.xslt'
    xmlLogoImage: file://D:\bb\trunk\Eclipse\testidea\si.isystem.itest.plugin.core\i
    cons\itest_window_128.gif
    xmlReportHeader: |-
      Test Report
    cssFile:  blue.css
    csvSeparator: ','
    isCSVHeaderLine: true
    isXLSVerticalHeader: false
    isIncludeTestSpec: true
    isAbsPathForLinks: true
    htmlViewMode: all
    testInfo:
      Date: 2016-05-16
      Time: 16:11:04
      Subversion rev.: 58135
      description: release build
      hardware: cortex
      wiWorkspacePath:
      bootloader: '1.05g'
      _appRev: '12.35'
  

    def isys_cmd_printHi(self, iyamlFile):
        print("HI! Script method executed, parameter = ", iyamlFile)