TraceTronic ECU-TEST1.18Minimum Jenkins requirement: 1.580.1ID: ecutest

Table of contents

Features

• Provides an easy integration and control of ECU-TEST and the Tool-Server with Jenkins
• Enables the execution of ECU-TEST packages and projects with their respective configurations

Moreover publishing test results is included in form of:

• Linking the TRF reports
• Generating various report formats
• Displaying test result trend graphs
• Uploading ATX test reports to TEST-GUIDE
• Parsing the ECU-TEST logs for warnings and errors

Configuration

ECU-TEST configuration

• ECU-TEST installations are administrated in the Jenkins system configuration or starting with Jenkins 2.0 in the global tool configuration at section "ECU-TEST"
• An installation entry is specified by an arbitrary name and the path to the installation directory
• The execution on a Jenkins slave requires the adaption of the ECU-TEST installation directory on the slave configuration page

TEST-GUIDE configuration

• TEST-GUIDE is also configured in the Jenkins system configuration at section "TEST-GUIDE"
• A new entry requires both an arbitrary name and the selection of a previously configured ECU-TEST installation
• Further settings like the server or ATX specific configuration can be found in the advanced block

Job configuration

The plugin provides several new build steps:

Note:

• All build steps have an "Advanced" button through which more options can be accessed
• All text fields can be parameterized and are resolved at build execution
  Syntax: $parameter or ${parameter}

The post-build actions are extended by these test report related post-build steps:

Job execution

• After job configuration is done a new build can be triggerd by clicking on the "Build now" button
• The console log contains detailed information per build step execution
• The test results are published both on the build page and the job page based on the latest build

Environment variables

• This plugin also exposes the tool and test related information of the current build as environment variables
• These variables are composed of $TT_<TOOL | TEST>_<information>_n where n is the 0-based index of the related build steps in the job configuration
• The summary of all generated environment variables is also shown on the build page

Screencasts

For more information about installation and configuration of the plugin as well as using the plugin in a slave-based setup see our screencasts.

Extensions

Job DSL

The Job DSL Plugin allows the programmatic creation of projects using a DSL. This plugin provides some specific DSL extensions.

steps {
  // Start ECU-TEST
  startET(String installation) {
    workspaceDir(String directory)
    settingsDir(String directory)
    timeout(int | String timeout = 120)
    debugMode(boolean debugMode = false)
    keepInstance(boolean keepInstance = false)
 }
  // Stop ECU-TEST
  stopET(String installation) {
    timeout(int | String timeout = 30)
  }
  // Start Tool-Server
  startTS(String installation) {
    toolLibsIni(String toolLibsPath)
    tcpPort(int | String port = 5017)
    timeout(int | String timeout = 120)
    keepInstance(boolean keepInstance = false)
 }
  // Stop Tool-Server
  stopTS(String installation)
    timeout(int | String timeout = 30)
  }
}

steps {
  // Run package
  testPackage(String packageFile) {
    testConfig {
      tbcFile(String tbcFile)
      tcfFile(String tcfFile)
      forceReload(boolean forceReload = false)
      loadOnly(boolean loadOnly = false)
      constants {
        constant(String name, String value)
        constant {
          name(String name)
          value(String value)
        }
      }
    }
    packageConfig {
      runTest(boolean runTest = true)
      runTraceAnalysis(boolean runTraceAnalysis = true)
      parameters {
        parameter(String name, String value)
        parameter {
          name(String name)
          value(String value)
        }
      }
    }
    executionConfig {
      timeout(int | String timeout = 3600)
      checkTestFile(boolean checkTestFile = true)
      stopOnError(boolean stopOnError = true)
    }
  }
  // Run project
  testProject(String projectFile) {
    testConfig { (...) }
    projectConfig {
      execInCurrentPkgDir(boolean currentPkgDir = false)
      filterExpression(String expression)
      // modes: NO_EXECUTION(0), SEQUENTIAL_EXECUTION(1), 
      // PARALLEL_EXECUTION(2), SEPARATE_SEQUENTIAL_EXECUTION(5), 
      // SEPARATE_PARALLEL_EXECUTION(6), NO_TESTCASE_EXECUTION(9)
      jobExecutionMode(int | String execMode = 1)
    }
    executionConfig { (...) }
  }
  // Run test folder
  testFolder(String testFolder) {
    // modes: PACKAGES_ONLY, PROJECTS_ONLY, PACKAGES_AND_PROJECTS
    scanMode(String mode = 'PACKAGES_AND_PROJECTS')
    recursiveScan(boolean recursive = false)
    testConfig { (...) }
    packageConfig { (...) }
    projectConfig { (...) }
    executionConfig { (...) }
  }
} 

steps {
  // Import projects
  importProjects {
    importFromArchive(String archivePath, String importPath, String importConfigPath, boolean replaceFiles)
    importFromArchive(String archivePath) {
      importPath(String importPath)
      importConfigPath(String importConfigPath)
      replaceFiles(boolean replaceFiles)
    }
    importFromTMS(String credentialsId, String projectPath, String importPath, int | String timeout = 60)
    importFromTMS(String credentialsId, String projectPath) {
      importPath(String importPath)
      timeout(int | String timeout = 60)
    }
    importFromTMSDir(String credentialsId, String projectDirPath, String importPath, int | String timeout = 60)
    importFromTMSDir(String credentialsId, String projectDirPath) {
      importPath(String importPath)
      timeout(int | String timeout = 60)
    }
    importAttributesFromTMS(String credentialsId, String projectPath, int | String timeout = 60)
    importAttributesFromTMS(String credentialsId, String projectPath) {
      timeout(int | String timeout = 60)
    }
  }  
  // Import packages
  importPackages {
    importFromTMS { (...) }
    importFromTMSDir { (...) }
    importAttributesFromTMS { (...) }
  }
  // Export projects
  exportProjects {
    exportToTMS(String credentialsId, String projectPath, String exportPath, boolean createNewPath, int | String timeout = 60)
    exportToTMS(String credentialsId, String projectPath) {
      exportPath(String exportPath)
      createNewPath(boolean createNewPath)
      timeout(int | String timeout = 60)
    }
    exportAttributesToTMS(String credentialsId, String projectPath, int | String timeout = 60)
    exportAttributesToTMS(String credentialsId, String projectPath) {
      timeout(int | String timeout = 60)
    }
  }  
  // Export packages
  exportPackages {
    exportToTMS { (...) }
    exportAttributesToTMS { (...) }
  }
}

publishers {
  // Publish ATX reports
  publishATX(String atxInstallation) {
    // Archive options
    allowMissing(boolean allowMissing = false)
    runOnFailed(boolean runOnFailed = false)
    archiving(boolean archiving = true)
    keepAll(boolean keepAll = true)
  }
  // Publish TRF reports
  publishTRF() {
    // Archive options...
  }
  // Publish UNIT reports
  publishUNIT(String installation) {
    unstableThreshold(double threshold = 0)
    failedThreshold(double threshold = 0)
    // Archive options...
  }
  // Publish ECU-TEST logs
  publishETLogs() {
    unstableOnWarning(boolean markUnstable = false)
    failedOnError(boolean markFailed = false)
    testSpecific(boolean testSpecific = false)
    // Archive options...
  }
  // Publish Report Generators
  publishGenerators(String installation) {
    generators {
      generator(String generatorName) {
        settings {
          setting(String name, String value)
          setting { 
            name(String name)
            value(String value)
          }
        }
      }
    }
    customGenerators { 
       customGenerator(String generatorName) { (...) }
    }
    // Archive options...
  }
  // Publish to Test Management System
  publishTMS(String installation, String credentialsId) {
    timeout(int | String timeout = 60)
    // Archive options...
  }
}

Pipeline

The Pipeline Plugin allows to orchestrate automation, simple or complex, using a DSL. This plugin supports the use of all provided build steps and post-build actions from within a Jenkins Pipeline build. The appropriate DSL syntax for these steps and actions can be easily generated with help of the Pipeline Snipped Generator using the related custom pipeline step or the 'step' command, when using plugin version prior to 1.13.

node('windows') {
    stage('Start Tools') {
        startET 'ECU-TEST'
    }
    stage('Test Execution') {
        testPackage testFile: 'sample.pkg', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
        testProject testFile: 'sample.prj', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
    }
    stage('Publish Reports') {
        publishATX 'TEST-GUIDE'
    }
    stage('Stop Tools') {
        stopET 'ECU-TEST'
    }
}

Issues

To report a bug or request an enhancement to this plugin please create a ticket in JIRA.

Known limitations

When using the plugin in a slave-based setup (especially running the slave at command line via JNLP) you also need to restart the slave when restarting the master.
This is due to already loaded libraries, for more information see this blog post and related issue JENKINS-31961.

Release 1.8 implemented an extension point that will workaround this issue. In order to work a new task has to be created in the Windows Task Scheduler named RESTART_JENKINS_SLAVE and configured with actions how to restart the slave.

Compatibility

• Jenkins LTS 1.580.1 or higher
• Java SE Runtime Environment 1.7 or higher
• ECU-TEST 6.3.0 or higher
• optional: TEST-GUIDE 1.11.0 or higher

License

This plugin is licensed under 3-clause BSD license.
More information can be found inside the LICENSE file.

Changelog

Release 1.18 (Jul 28, 2017)

Added option to re-use ECU-TEST and Tool-Server instances

Ignore ECU-TEST patch version in compatibility checks
Check architecture compatibility between ECU-TEST and JVM

Updated available ATX settings to TEST-GUIDE 1.46.0

Ignore test configuration validation if previous ones re-used

Release 1.17.1 (Jun 06, 2017)

Fixed duplicate test project action

Release 1.17 (Jun 02, 2017)

Added option to keep currently loaded configurations
Added option to import missing packages for project imports
Added build step to import and export packages/projects from/to test management system
Added build step to import and export package/project attributes from/to test management system
Added appropriate Job DSL and Pipeline plugin support for for test management build steps

Keep test results in case of manual build abort or exceeded timeout
Debug log used COM ProgId and version (use system property ecutest.debugLog=<true|false>)
Retain backward compatibility for legacy test management builders

Ensured compatibility up to ECU-TEST 6.6.0
Updated available ATX settings to TEST-GUIDE 1.44.0

Release 1.16 (Mar 10, 2017)

Improved summary view for tool and test related build variables
Added support for multiple TEST-GUIDE projects

Append to existing JUnit results to prevent duplicated trend graphs
Exclude job analysis files when traversing sub reports
Compatibility changes for TEST-GUIDE 1.39+

Updated available ATX settings to TEST-GUIDE 1.41.0

Release 1.15 (Jan 06, 2017)

Added build step to import projects from an archive or a test management system (like RQM, ALM)
Added post-build step allowing to export test results to a test management system
Added appropriate Job DSL and Pipeline plugin support for new features above

Depend on Icon Shim Plugin to simplify icon usage (JENKINS-36472)

Fixed ATX report links for long-running uploads

Release 1.14 (Nov 25, 2016)

Remove empty installations when saving configurations
Avoid circular dependency warning on Jenkins startup (JENKINS-39169)

Fixed TRF include/exclude pattern for report publishers
Fixed ATX report links for sub project execution

Updated available ATX settings to TEST-GUIDE 1.38.0

Release 1.13 (Oct 14, 2016)

Added @Symbol annotation to simplify pipeline syntax

Exclude job analysis reports from publishing

Release 1.12 (Sep 16, 2016)

Added support for new TRF naming convention (like package/project name)
Added support for changed installation path of Tool-Server
Added support for parallel installations (configurable COM ProgID)

Properly log ATX report upload errors
Simplified configuration with automatic migration
Improved Windows process control (using WinP)

Fixed absolute configuration path check

Ensured compatibility up to ECU-TEST 6.5.0
Updated available ATX settings to TEST-GUIDE 1.35.0

Release 1.11 (May 27, 2016)

Added support for Pipeline plugin (fka Workflow) (JENKINS-31999)

Bumped Jenkins baseline to LTS 1.580.1

Release 1.10 (May 13, 2016)

Added option to publish and parse test specific log files

Allow use of parameterized tool selection in Job DSL extensions

Bumped to new parent POM 2.9
Updated available ATX settings to TEST-GUIDE 1.30.0

Release 1.9 (Apr 15, 2016)

Added publishing option to disable archiving reports
Added option to keep artifacts of most recent build only (project level)

Ensured compatibility up to ECU-TEST 6.4.0
Updated available ATX settings to TEST-GUIDE 1.29.0

Release 1.8 (Mar 18, 2016)

Added workaround for already loaded JACOB library (JENKINS-31961)
Added report publisher allowing to execute report generators

Fixed test execution handling when aborting the build (JENKINS-33457)

Release 1.7 (Mar 04, 2016)

Added parameterized ECU-TEST and TEST-GUIDE selection
Allow use of parameterized ATX settings

Improved test execution timeout handling

Ensured compatibility up to ECU-TEST 6.3.2
Updated available ATX settings to TEST-GUIDE 1.27.0

Release 1.6 (Jan 29, 2016)

Added option to only load the configurations without starting them automatically

Check for existing global constants, prevents to unnecessarily reload the configurations

Release 1.5 (Dec 18, 2015)

Added option to explicitly set the ECU-TEST settings directory

Allow use of GStrings in Job DSL extensions
Discard empty global constants and package parameters

Fixed undetected ATX upload errors (JENKINS-32078)

Release 1.4 (Dec 04, 2015)

Added support to pre-check packages and projects
Added option to force reloading the current configuration

Highlighted the test result in the console log

Fixed ATX report links for multiple archived ATX files (JENKINS-31770)

Release 1.3 (Nov 23, 2015)

Added post-execution timeouts

Improved Job DSL extensions by providing default settings

Fixed form validation for some ATX settings (JENKINS-31703)

Release 1.2 (Nov 06, 2015)

Added support for Job DSL plugin
Added ZIP download link for published TRF and ATX reports

Aggregated UNIT test results for matrix projects

Release 1.1 (Oct 23, 2015)

Added support for separate sub-project execution
Added archiving of ATX reports for disabled ATX upload

Improved publisher behavior handling ECU-TEST instances
Improved ECU-TEST log parser performance
Improved validation of TEST-GUIDE server availability

Release 1.0 (Oct 09, 2015)

Initial release