mabl

This plugin allows easy launching of mabl tests as a step in your Jenkins build. Your Jenkins build outcome will be tied to that of your deployment event.

• Plugin Installation
• Features
• Requirements
• Adding a mabl project step
• Adding a mabl pipeline step
• Environment variable collection
• Configuring a proxy
• Upgrading from previous versions
• Change log
• Building from source
• Local development
  • Jenkins docker
  • Local machine
• Deployment

Install the plugin into your Jenkins v2.387.3+ server from the Available Plugins tab by searching for "mabl".

• Launch mabl tests as a project step
• Fail your Jenkins build if the mabl tests fail
  • Optionally continue on failure

• Minimum Jenkins version: 2.387.3
• Minimum Java runtime version: 11
• mabl API key
  • See integration docs for details

• Add a new credential accessible to the Jenkins job using the Secret text credential kind. The scope of the credential must be Global. The credential may be placed in any of the credential domains.
• Set the secret to to the mabl API key
• Set the ID of the credential to a value of your choosing

Add a Run mabl tests step to your new or existing project.

Select the API key from the drop-down list. Then select environment, and application from the drop-down. At least one of environment and application must be supplied.  All matching plans and their tests will be run during this step. The step will block until the tests are complete/failed, or one hour has elapsed.

The Advanced... button opens up additional options:

• Filter by plan labels to run only plans tagged with the selected labels. (You can select multiple labels using shift/ctrl click.)
• Specify the test branch of your mabl tests
• Continue execution if your plan/tests fail
• Continue execution if there is an unexpected error in the mabl API
• Disable SSL verification

In a pipeline step, select 'Configure'. In the pipeline script, insert a pipeline step for mabl. This can be written by hand or created via the 'Pipeline Syntax' tool. The available parameters are:

• applicationId: Selects the application to run deployments against (one of environmentId or applicationId is required)
• continueOnMablError: Continue if there is an unexpected error in the mabl API
• continueOnPlanFailure: Continue if your plan/tests fail
• environmentId: Selects the environment to run deployments against (one of environmentId or applicationId is required)
• restApiKeyId: The id of the API key secret of the desired deployment workspace. The id is the value that is assigned in the Jenkins configuration to the corresponding secret. It is not the secret itself. Required
• labels: if specified, only plans with (any of the) labels will be triggered. If multiple labels are provided, separate them with commas.
• mablBranch: if specified, then tests from the specified branch will be executed. You may also want to use a Jenkins environment variable to specify the branch.

Note that if you do not want to specify either the application ID or the environment ID, then use an empty value in the pipeline configuration instead of completely omitting the corresponding configuration attributes.

The following sections shows how to use the integration plugin in either a declarative or in a scripted pipeline.

mabl applicationId: 'APP-ID-a', continueOnMablError: true, continueOnPlanFailure: true, environmentId: 'ENV-ID-e', restApiKeyId: 'REST-API-KEY-ID'

node {
  stage('mabl') {
    step([$class: 'MablStepBuilder', restApiKeyId: 'REST-API-KEY-ID', environmentId: 'ENV-ID-e', applicationId: 'APP-ID-a'])
  }
}

Note that Jenkins pipelines may time out before your tests complete. In this case, adjust or set the timeout option in your pipeline configuration. See the Available Options section on the Jenkins documentation site for more information.

The mabl plugin will optionally collect build and repository information to send along with mabl deployments. This will give more information and insights into testing environment state for test runs. We will not collect sensitive information such as passwords or API keys.

To enable this feature:

• Go to the 'Manage Jenkins' page from the 'Dashboard'
• Go to the 'Configure System' page
• Find the 'mabl' section and check the 'Send build environment variables to mabl' checkbox

Now you will see the environment and build information collected in the output log of future runs that use a mabl plugin step.

Example Variable Collections

Send build environment variables is set. Collecting the following information:
  'GIT_BRANCH' => 'origin/main'
  'GIT_COMMIT' => 'a246e0c756791965bdb6fc9caa1a36775422fcec'
  'GIT_URL' => 'git@github.com:example/my_repository.git'
  'GIT_PREVIOUS_COMMIT' => 'a246e0c73a691935bdb6fc9caa1a36775422fce5'
  'JOB_NAME' => 'mabl integration'
  'BUILD_NUMBER' => '47'
  'RUN_DISPLAY_URL' => 'http://127.0.0.1:9090/job/mabl%20integration/47/display/redirect'

This plugin requires access to resources hosted on the mabl cloud. If your Jenkins instance cannot directly access Internet resources, then you have to configure Jenkins to use a proxy.

1. Visit the Manage Jenkins and then to the Manage Plugins page
2. Click on the Advanced tab
3. Fill out the Server, Port, and - if authentication is needed - the User name and/or Password fields that reflect the proxy configuration in your environment

Note that the format of the labels configuration setting has changed. When editing an existing job with labels configured, make sure that you update the value to be a comma-separated list of values.

Note that

• Jenkins versions 1.x are no longer supported
• Minimum required Java version is Java 11
• Jobs with mabl steps will have to be manually updated
  1. Create a Secret text kind credential to store the mabl API key. The scope of the credential must be set to Global. The credential may be placed in any of the credential domains.
  2. Make sure that you set the ID of the credentials
  3. Update the mabl step in each affected job

• Update documentation
• Increased minimum Jenkins version

• Fix broken documentation link
• Fix continueOnPlanFailure

• Patched security vulnerabilities
• Increased minimum Jenkins version

• Replace usages of deprecated imports

• Updated credentials plugin dependency to address security vulnerability
• Update README to clarify set up and validation steps

• Increased timeout to trigger large deployments
• Increased timeout for maximum test run duration
• Updated documentation on increasing default Jenkins pipeline timeout

• Updated documentation to clarify how to omit application ID or environment ID from pipelines
• Updated GSON, SpotBugs and Jenkins Maven HPI plugin dependencies
• Remove dependence on Jenkins Base64 class

• Fixed an issue where the associated test cases IDs were incorrectly recorded in the generated JUnit report
• Updated WireMock, JUnit, GSON, SpotBugs dependencies
• Address warnings on new SpotBugs findings

• Changed minimum Jenkins version to 2.222.4 (pre-requisite for updating credentials plugin dependency)
• Updated credentials plugin dependency to address security vulnerability
• Updated plain-credentials, wiremock dependencies

• Increased request timeout for interacting with mabl API
• Added retry handler for idempotent operations when interacting with mabl API
• Update components: Wiremock, JUnit5, GSON, Spotbugs, JaCoCo

• Added support for running plugin with Java Runtime Environment 11
• Added GSON library for deserializing from JSON
• Replaced Jackson with XStream for XML serialization
• Updated pipeline sample and clarified how to use REST API keys

• Added support for using forward proxies for interacting with mabl
• Updated JUnit, Mockito, JSON, Jackson, Spotbugs dependencies

• Updated Apache HTTP client, Mockito, Jackson, Jenkins HPI plugin, and Spotbugs dependencies

• Updated Wiremock, JUnit, Jackson, Spotbugs and JaCoCo dependencies
• Added validation for branch field
• Added trimming whitespaces from labels
• Removed use of deprecated API endpoint

• Set explicit dependency on credentials and plain credentials dependency to address incompatibility issue with Jenkins Enterprise

• Added JaCoCo test coverage report and checks
• Updated JUnit, Mockito dependency
• Added unit test for branching
• Revised Jenkins BOM dependency for compatibility reasons

• Fixed a regression related to default endpoints

• Updated JUnit report to include information about test case IDs in both the properties section of the TestSuite tag as well as in the non-standard properties section of the TestCase tag following the format used by the XRay application
• Updated JUnit report to include information about the number of skipped tests. The report file now also includes an empty Skipped element inside the corresponding TestCase-s.
• Fixed code to properly populate the execution time for tests
• Updated code to use main instead of master
• Changed plan label configuration to use a single text box to avoid a problem related to inadvertently clearing out existing values on save
• Updated mockito, wiremock, jackson, maven-hpi-plugin and spotbugs dependencies
• Enabled using alternate mabl endpoints for internal testing purpose
• Updated test output to indicate when a plan is retried

• Added a configuration option to execute mabl tests in a branch
• Updated mockito core dependency and marked it for test scope only

• Any plan with the retry option enabled used to fail the build when the first attempt failed even when the retry succeeded
• Fixed an issue when a stopped plan run caused a null pointer exception
• Updated the test output to include a link to the deployment event in the mabl app

• Updated Jackson dependencies to 2.11.0

• Adjusted labels in configuration
• Set compatibility version
• 0.0.24 was never released

• Adjusted minimum version to 2.164.3

• Fix for a benign exception visible in the Jenkins logs

• Raised minimum version to 2.121.3
• Switched to use Secret text credential kind for storing mabl API key

• Moved API key to credentials plugin. Existing users must update their configuration manually.
• Minimum Java runtime requirement is now Java 7
• Minimum supported Jenkins version is 1.625.1
• Fixed incompatibility with Pipeline jobs
• Replaced FindBugs plugin with SpotBugs

• Security fixes for handling API key
• Fixed POM to use Findbugs version variable
• Improved the plan label form by displaying the currently configured values

• Screenshot and documentation updates
• Updates to dependencies

• API key is now encrypted in configuration file
• Updated messages to replace journey with test

• Fix bug when displaying status for multi-stage plans.

• The output of failed run with link is now available

• Support for plan label filtering in the mabl build step

• The mabl plugin is now available to use a pipeline step
• Optionally allow mabl to collect environment information to enhance deployment journey insights

• The output of mabl tests are now available as junit XML in the workspace directory for reports and download  

• Environment and Application drop downs automatically populated from API key

• More resilient to failures

• Advanced form validation support added

• Support for Windows console output and encodings

• Initial release

1. Clone this repo
2. build with mvn clean package
3. Copy the plugin in target/mabl-integration.hpi to your Jenkins plugins/ directory
4. Restart Jenkins

You can also install the .hpi file from the web UI by visting Jenkins > Manage Jenkins > Manager Plugins > Advanced > Upload Plugin.

Overview of how to launch a Jenkins Docker instance with Jenkins, then build the plugin and deploy it that instance.

Note: Please check Jenkins Hub for any specific image version before pulling Jenkins image.

# Launch Jenkins container and automatically pull the image if not present
docker run -d -p 9090:8080 --name=jenkins jenkins/jenkins

# Find initialAdminPassword to unlock Jenkins on a browser at localhost:9090
docker exec -it jenkins bash -c "cat /var/jenkins_home/secrets/initialAdminPassword"

# Setup your Jenkins instance

# Build and deploy plugin to Jenkins (make sure you're in the mabl-integration-plugin directory)
mvn clean package \
  && docker cp target/mabl-integration.hpi jenkins:/var/jenkins_home/plugins/ \
  && docker restart jenkins

To ensure compatibility with Jenkins Enterprise, make sure the credentials and plain credentials dependencies are compatible in the minimum version of the enterprise release. Check the plugin list in the CloudBees Jenkins Distribution Release Notes.

You can also test with Jenkins Enterprise using the images published on Docker Hub.

# Launch Jenkins Enterprise container and automatically pull the image if not present
docker run -d -p 9090:8080 --name=jenkins cloudbees/jenkins-enterprise

# Find initialAdminPassword to unlock Jenkins on a browser at localhost:9090
docker exec -it jenkins bash -c "cat /var/jenkins_home/secrets/initialAdminPassword"

# Setup your Jenkins instance

# Build and deploy plugin to Jenkins (make sure you're in the mabl-integration-plugin directory)
mvn clean package \
  && docker cp target/mabl-integration.hpi jenkins:/var/jenkins_home/plugins/ \
  && docker restart jenkins

If the plugin is not loaded due to dependency errors, try restarting Jenkins.

docker restart jenkins

Overview of how to run/debug plugin with Intellij on local machine.

• Open the project in Intellij

• Look for Edit Configurations under Run

• Add Maven template

• Configure your working directory to point to your project path similarly as the screenshot below:

• Click OK and then run/debug the plugin

• Wait until you see

    INFO: Jenkins is fully up and running
  

• Log into this URL on your web browser

    localhost:9090/jenkins
  

Now you can test your plugin.

Before making a new plugin release, ensure code is in high quality, fully tested state. See extra checks.

1. Make sure you bump pom.xml version with -SNAPSHOT in your PR before merging to main
2. Update your ~/.m2/settings.xml according to the Jenkins docs.
3. Setup and run a GitHub ssh agent.
4. Run mvn release:prepare release:perform -B from the HEAD of main
5. Run mvn deploy on success of above step.

Wait ~8 hours for plugin to become GA across all Jenkins instances under the "Available Plugins" listing.