×
Find plugins

TraceTronic ECU-TEST1.18Minimum Jenkins requirement: 1.580.1ID: ecutest

This plugin integrates Jenkins with ECU-TEST and generates reports on automated test execution.
 

ECU-TEST is a test automation software for the validation of embedded systems in automotive environments developed by TraceTronic GmbH.
This software executes regression tests which are essential for validating complex technical products such as electronic control units (ECUs).
It supports standardized access to a broad range of test tools and provides an extensive automation of distributed test environments (SiL – MiL – HiL – vehicle).

Table of contents

Features

  • Provides an easy integration and control of ECU-TEST and the Tool-Server with Jenkins
  • Enables the execution of ECU-TEST packages and projects with their respective configurations

Moreover publishing test results is included in form of:

  • Linking the TRF reports
  • Generating various report formats
  • Displaying test result trend graphs
  • Uploading ATX test reports to TEST-GUIDE
  • Parsing the ECU-TEST logs for warnings and errors

Configuration

ECU-TEST configuration

  • ECU-TEST installations are administrated in the Jenkins system configuration or starting with Jenkins 2.0 in the global tool configuration at section "ECU-TEST"
  • An installation entry is specified by an arbitrary name and the path to the installation directory
  • The execution on a Jenkins slave requires the adaption of the ECU-TEST installation directory on the slave configuration page

TEST-GUIDE configuration

  • TEST-GUIDE is also configured in the Jenkins system configuration at section "TEST-GUIDE"
  • A new entry requires both an arbitrary name and the selection of a previously configured ECU-TEST installation
  • Further settings like the server or ATX specific configuration can be found in the advanced block

Job configuration

The plugin provides several new build steps:

Note:

  • All build steps have an "Advanced" button through which more options can be accessed
  • All text fields can be parameterized and are resolved at build execution
    Syntax: $parameter or ${parameter}

The post-build actions are extended by these test report related post-build steps:

Job execution

  • After job configuration is done a new build can be triggerd by clicking on the "Build now" button
  • The console log contains detailed information per build step execution
  • The test results are published both on the build page and the job page based on the latest build

Environment variables

  • This plugin also exposes the tool and test related information of the current build as environment variables
  • These variables are composed of $TT_<TOOL | TEST>_<information>_n where n is the 0-based index of the related build steps in the job configuration
  • The summary of all generated environment variables is also shown on the build page
  Tool related variables

Environment variable

Description

TT_TOOL_NAME

Name of the selected ECU-TEST installation

TT_TOOL_INSTALL

Full path to related ECU-TEST executable

TT_TOOL_VERSION

Program version of selected ECU-TEST installation

TT_TOOL_WORKSPACE

ECU-TEST workspace directory

TT_TOOL_SETTINGS

ECU-TEST settings directory

TT_TOOL_DEBUG

ECU-TEST debug mode

TT_TOOL_TIMEOUT

Maximum execution time

  Test related variables

Environment variable

Description

TT_TEST_NAME

Name of the executed test

TT_TEST_FILE

Full path to related test file

TT_TEST_TYPE

Type of related test (PACKAGE, PROJECT)

TT_TEST_DESCRIPTION

Test description

TT_TEST_TBC

Full path to test bench configuration file

TT_TEST_TCF

Full path to test configuration file

TT_TEST_TIMEOUT

Maximum execution time

TT_TEST_RESULT

Test result (SUCCESS, INCONCLUSIVE, FAILED, ERROR, NONE)

TT_TEST_REPORT

Full path to test report directory

Screencasts

For more information about installation and configuration of the plugin as well as using the plugin in a slave-based setup see our screencasts.

Extensions

Job DSL

The Job DSL Plugin allows the programmatic creation of projects using a DSL. This plugin provides some specific DSL extensions.

  Controlling tools
steps {
  // Start ECU-TEST
  startET(String installation) {
    workspaceDir(String directory)
    settingsDir(String directory)
    timeout(int | String timeout = 120)
    debugMode(boolean debugMode = false)
    keepInstance(boolean keepInstance = false)
 }
  // Stop ECU-TEST
  stopET(String installation) {
    timeout(int | String timeout = 30)
  }
  // Start Tool-Server
  startTS(String installation) {
    toolLibsIni(String toolLibsPath)
    tcpPort(int | String port = 5017)
    timeout(int | String timeout = 120)
    keepInstance(boolean keepInstance = false)
 }
  // Stop Tool-Server
  stopTS(String installation)
    timeout(int | String timeout = 30)
  }
}
  Test execution
steps {
  // Run package
  testPackage(String packageFile) {
    testConfig {
      tbcFile(String tbcFile)
      tcfFile(String tcfFile)
      forceReload(boolean forceReload = false)
      loadOnly(boolean loadOnly = false)
      constants {
        constant(String name, String value)
        constant {
          name(String name)
          value(String value)
        }
      }
    }
    packageConfig {
      runTest(boolean runTest = true)
      runTraceAnalysis(boolean runTraceAnalysis = true)
      parameters {
        parameter(String name, String value)
        parameter {
          name(String name)
          value(String value)
        }
      }
    }
    executionConfig {
      timeout(int | String timeout = 3600)
      checkTestFile(boolean checkTestFile = true)
      stopOnError(boolean stopOnError = true)
    }
  }
  // Run project
  testProject(String projectFile) {
    testConfig { (...) }
    projectConfig {
      execInCurrentPkgDir(boolean currentPkgDir = false)
      filterExpression(String expression)
      // modes: NO_EXECUTION(0), SEQUENTIAL_EXECUTION(1), 
      // PARALLEL_EXECUTION(2), SEPARATE_SEQUENTIAL_EXECUTION(5), 
      // SEPARATE_PARALLEL_EXECUTION(6), NO_TESTCASE_EXECUTION(9)
      jobExecutionMode(int | String execMode = 1)
    }
    executionConfig { (...) }
  }
  // Run test folder
  testFolder(String testFolder) {
    // modes: PACKAGES_ONLY, PROJECTS_ONLY, PACKAGES_AND_PROJECTS
    scanMode(String mode = 'PACKAGES_AND_PROJECTS')
    recursiveScan(boolean recursive = false)
    testConfig { (...) }
    packageConfig { (...) }
    projectConfig { (...) }
    executionConfig { (...) }
  }
} 
  Test management
steps {
  // Import projects
  importProjects {
    importFromArchive(String archivePath, String importPath, String importConfigPath, boolean replaceFiles)
    importFromArchive(String archivePath) {
      importPath(String importPath)
      importConfigPath(String importConfigPath)
      replaceFiles(boolean replaceFiles)
    }
    importFromTMS(String credentialsId, String projectPath, String importPath, int | String timeout = 60)
    importFromTMS(String credentialsId, String projectPath) {
      importPath(String importPath)
      timeout(int | String timeout = 60)
    }
    importFromTMSDir(String credentialsId, String projectDirPath, String importPath, int | String timeout = 60)
    importFromTMSDir(String credentialsId, String projectDirPath) {
      importPath(String importPath)
      timeout(int | String timeout = 60)
    }
    importAttributesFromTMS(String credentialsId, String projectPath, int | String timeout = 60)
    importAttributesFromTMS(String credentialsId, String projectPath) {
      timeout(int | String timeout = 60)
    }
  }  
  // Import packages
  importPackages {
    importFromTMS { (...) }
    importFromTMSDir { (...) }
    importAttributesFromTMS { (...) }
  }
  // Export projects
  exportProjects {
    exportToTMS(String credentialsId, String projectPath, String exportPath, boolean createNewPath, int | String timeout = 60)
    exportToTMS(String credentialsId, String projectPath) {
      exportPath(String exportPath)
      createNewPath(boolean createNewPath)
      timeout(int | String timeout = 60)
    }
    exportAttributesToTMS(String credentialsId, String projectPath, int | String timeout = 60)
    exportAttributesToTMS(String credentialsId, String projectPath) {
      timeout(int | String timeout = 60)
    }
  }  
  // Export packages
  exportPackages {
    exportToTMS { (...) }
    exportAttributesToTMS { (...) }
  }
}
  Publishing reports
publishers {
  // Publish ATX reports
  publishATX(String atxInstallation) {
    // Archive options
    allowMissing(boolean allowMissing = false)
    runOnFailed(boolean runOnFailed = false)
    archiving(boolean archiving = true)
    keepAll(boolean keepAll = true)
  }
  // Publish TRF reports
  publishTRF() {
    // Archive options...
  }
  // Publish UNIT reports
  publishUNIT(String installation) {
    unstableThreshold(double threshold = 0)
    failedThreshold(double threshold = 0)
    // Archive options...
  }
  // Publish ECU-TEST logs
  publishETLogs() {
    unstableOnWarning(boolean markUnstable = false)
    failedOnError(boolean markFailed = false)
    testSpecific(boolean testSpecific = false)
    // Archive options...
  }
  // Publish Report Generators
  publishGenerators(String installation) {
    generators {
      generator(String generatorName) {
        settings {
          setting(String name, String value)
          setting { 
            name(String name)
            value(String value)
          }
        }
      }
    }
    customGenerators { 
       customGenerator(String generatorName) { (...) }
    }
    // Archive options...
  }
  // Publish to Test Management System
  publishTMS(String installation, String credentialsId) {
    timeout(int | String timeout = 60)
    // Archive options...
  }
}

Pipeline

The Pipeline Plugin allows to orchestrate automation, simple or complex, using a DSL. This plugin supports the use of all provided build steps and post-build actions from within a Jenkins Pipeline build. The appropriate DSL syntax for these steps and actions can be easily generated with help of the Pipeline Snipped Generator using the related custom pipeline step or the 'step' command, when using plugin version prior to 1.13.

  Basic pipeline example
node('windows') {
    stage('Start Tools') {
        startET 'ECU-TEST'
    }
    stage('Test Execution') {
        testPackage testFile: 'sample.pkg', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
        testProject testFile: 'sample.prj', testConfig: [tbcFile: 'sample.tbc', tcfFile: 'sample.tcf']
    }
    stage('Publish Reports') {
        publishATX 'TEST-GUIDE'
    }
    stage('Stop Tools') {
        stopET 'ECU-TEST'
    }
}

Issues

To report a bug or request an enhancement to this plugin please create a ticket in JIRA.

T Key Summary P Status
Loading...
Refresh

Known limitations

When using the plugin in a slave-based setup (especially running the slave at command line via JNLP) you also need to restart the slave when restarting the master.
This is due to already loaded libraries, for more information see this blog post and related issue JENKINS-31961.

Release 1.8 implemented an extension point that will workaround this issue. In order to work a new task has to be created in the Windows Task Scheduler named RESTART_JENKINS_SLAVE and configured with actions how to restart the slave.

FAQ

tbd

Compatibility

  • Jenkins LTS 1.580.1 or higher
  • Java SE Runtime Environment 1.7 or higher
  • ECU-TEST 6.3.0 or higher
  • optional: TEST-GUIDE 1.11.0 or higher

License

This plugin is licensed under 3-clause BSD license.
More information can be found inside the LICENSE file.

Changelog

Release 1.18 (Jul 28, 2017)

(plus) Added option to re-use ECU-TEST and Tool-Server instances

(tick) Ignore ECU-TEST patch version in compatibility checks
(tick) Check architecture compatibility between ECU-TEST and JVM

(info) Updated available ATX settings to TEST-GUIDE 1.46.0

(error) Ignore test configuration validation if previous ones re-used

Release 1.17.1 (Jun 06, 2017)

(error) Fixed duplicate test project action

Release 1.17 (Jun 02, 2017)

(plus) Added option to keep currently loaded configurations
(plus) Added option to import missing packages for project imports
(plus) Added build step to import and export packages/projects from/to test management system
(plus) Added build step to import and export package/project attributes from/to test management system
(plus) Added appropriate Job DSL and Pipeline plugin support for for test management build steps

(tick) Keep test results in case of manual build abort or exceeded timeout
(tick) Debug log used COM ProgId and version (use system property ecutest.debugLog=<true|false>)
(tick) Retain backward compatibility for legacy test management builders

(info) Ensured compatibility up to ECU-TEST 6.6.0
(info) Updated available ATX settings to TEST-GUIDE 1.44.0

Release 1.16 (Mar 10, 2017)

(plus) Improved summary view for tool and test related build variables
(plus) Added support for multiple TEST-GUIDE projects

(tick) Append to existing JUnit results to prevent duplicated trend graphs
(tick) Exclude job analysis files when traversing sub reports
(tick) Compatibility changes for TEST-GUIDE 1.39+

(info) Updated available ATX settings to TEST-GUIDE 1.41.0

Release 1.15 (Jan 06, 2017)

(plus) Added build step to import projects from an archive or a test management system (like RQM, ALM)
(plus) Added post-build step allowing to export test results to a test management system
(plus) Added appropriate Job DSL and Pipeline plugin support for new features above

(tick) Depend on Icon Shim Plugin to simplify icon usage (JENKINS-36472)

(error) Fixed ATX report links for long-running uploads

Release 1.14 (Nov 25, 2016)

(tick) Remove empty installations when saving configurations
(tick) Avoid circular dependency warning on Jenkins startup (JENKINS-39169)

(error) Fixed TRF include/exclude pattern for report publishers
(error) Fixed ATX report links for sub project execution

(info) Updated available ATX settings to TEST-GUIDE 1.38.0

Release 1.13 (Oct 14, 2016)

(plus) Added @Symbol annotation to simplify pipeline syntax

(error) Exclude job analysis reports from publishing

Release 1.12 (Sep 16, 2016)

(plus) Added support for new TRF naming convention (like package/project name)
(plus) Added support for changed installation path of Tool-Server
(plus) Added support for parallel installations (configurable COM ProgID)

(tick) Properly log ATX report upload errors
(tick) Simplified configuration with automatic migration
(tick) Improved Windows process control (using WinP)

(error) Fixed absolute configuration path check

(info) Ensured compatibility up to ECU-TEST 6.5.0
(info) Updated available ATX settings to TEST-GUIDE 1.35.0

Release 1.11 (May 27, 2016)

(plus) Added support for Pipeline plugin (fka Workflow) (JENKINS-31999)

(info) Bumped Jenkins baseline to LTS 1.580.1

Release 1.10 (May 13, 2016)

(plus) Added option to publish and parse test specific log files

(tick) Allow use of parameterized tool selection in Job DSL extensions

(info) Bumped to new parent POM 2.9
(info) Updated available ATX settings to TEST-GUIDE 1.30.0

Release 1.9 (Apr 15, 2016)

(plus) Added publishing option to disable archiving reports
(plus) Added option to keep artifacts of most recent build only (project level)

(info) Ensured compatibility up to ECU-TEST 6.4.0
(info) Updated available ATX settings to TEST-GUIDE 1.29.0

Release 1.8 (Mar 18, 2016)

(plus) Added workaround for already loaded JACOB library (JENKINS-31961)
(plus) Added report publisher allowing to execute report generators

(error) Fixed test execution handling when aborting the build (JENKINS-33457)

Release 1.7 (Mar 04, 2016)

(plus) Added parameterized ECU-TEST and TEST-GUIDE selection
(plus) Allow use of parameterized ATX settings

(tick) Improved test execution timeout handling

(info) Ensured compatibility up to ECU-TEST 6.3.2
(info) Updated available ATX settings to TEST-GUIDE 1.27.0

Release 1.6 (Jan 29, 2016)

(plus) Added option to only load the configurations without starting them automatically

(tick) Check for existing global constants, prevents to unnecessarily reload the configurations

Release 1.5 (Dec 18, 2015)

(plus) Added option to explicitly set the ECU-TEST settings directory

(tick) Allow use of GStrings in Job DSL extensions
(tick) Discard empty global constants and package parameters

(error) Fixed undetected ATX upload errors (JENKINS-32078)

Release 1.4 (Dec 04, 2015)

(plus) Added support to pre-check packages and projects
(plus) Added option to force reloading the current configuration

(tick) Highlighted the test result in the console log

(error) Fixed ATX report links for multiple archived ATX files (JENKINS-31770)

Release 1.3 (Nov 23, 2015)

(plus) Added post-execution timeouts

(tick) Improved Job DSL extensions by providing default settings

(error) Fixed form validation for some ATX settings (JENKINS-31703)

Release 1.2 (Nov 06, 2015)

(plus) Added support for Job DSL plugin
(plus) Added ZIP download link for published TRF and ATX reports

(tick) Aggregated UNIT test results for matrix projects

Release 1.1 (Oct 23, 2015)

(plus) Added support for separate sub-project execution
(plus) Added archiving of ATX reports for disabled ATX upload

(tick) Improved publisher behavior handling ECU-TEST instances
(tick) Improved ECU-TEST log parser performance
(tick) Improved validation of TEST-GUIDE server availability

Release 1.0 (Oct 09, 2015)

(info) Initial release

ArchivesGet past versions
Labels