×
Find plugins

Pretested Integration2.4.1Minimum Jenkins requirement: 1.580ID: pretested-integration

Developed by

Sponsored by

Introduction

The Pretested Integration Plugin offers a branchy approach to pretested integration (also known as pre-tested commits), which upholds the invariant; that for a specific branch, known as the integration branch, all commits have been verified.
This plugin is developed by Praqma and sponsored by Atmel - It's maintained in the scope of Joint Open Source Roadmap Alliance (JOSRA).

The plugin relies on the SCM plugin to establish the workspace and then takes over to do the integration. Finally the job makes the decision whether to like and commit the result or hate and discard it. To get a comprehensive introduction to how the plugin relies on the SCM plugin please read the blog post at the JOSRA: Pretested Integration Plugin.

Features

  • If you deliver (git push) more than one commit on the ready branch it will be delivered to the integration branch using the strategy.
  • Two strategies are supported: squashed (using git --squash merges) and accumulated (using git --no-ff merges).
  • If you deliver one commit that can be fast-forwarded, it will be.
  • Credentials support for the SCM configuration.

Limitations, requirements and dependencies

  • The plugin only supports the Jenkins Freestyle project (not Maven or Multi-configuration aka. Matrix). In other project types than the Freestyle, the UI of Pretested Integration will not show up.
  • Git configuration dependencies: the user account of the Jenkins build executor must be available to push to your integration repository. Either use the Git SCM credential configuration in the Jenkins job, or use either http- or ssh-credentials configured for the user account running the Jenkins slave.

References

  • For more background information and discussions on the different merge strategies available please read the blog post at the JOSRA: Pretested Integration Plugin.
  • For a paper on how to implement at complete flow of automated continuous delivery - including pretested integration - read the white paper: An Automated Git Branching Flow
  • To follow the roadmap for this plugin see the Trello board.
  • Developer oriented documentation is found in the repository readme.

Support and contact

Please send us an email on support@praqma.net if you have questions regarding the plugin.

If you find an issue or have feature requests, use the Jenkins issue tracker.

Comments and discussions on this page might not always be noticed.

The recommended setup and git workflow

Here is a simple git workflow where you can work on a features branch, and when ready push to a ready-branch. The Pretested Integration plugin, if configured as described will then pick up your changes, merge them and verify them. If verified they are integrated on the integration branch. The ready branch are automatically deleted if integration was successful.

Recommendation:

  • Use one repository in your job configuration - the integration repository. Avoid using several repositories - model your dependencies in other (better) ways.
  • Name your repository origin
  • Use master as integration branch (destination).
  • Use origin/ready/** as specifier for ready branches - only branches matching this expression will trigger the build

The simple plugin configuration

To configure the plugin to pick up ready branches, verify them and deliver them to the integration branch use the following configuration:


Configure the SCM as above.

  • Recommended ready branch specifier in 'Branch Specifier' to be origin/ready/** (note the origin matches repository 'Name' field)
  • Remember to add the two additional behaviours to ensure a clean work space and that old deleted branches are cleaned
  • It is recommended to name your repository in the 'Name' field. In this example we use origin.


Configure the Pretested Integration plugin with integration branch master and the named repository origin.
There is a Pretested Integration plugin post-build step, which ensure verified changes are published, this can be added in a later step.

Now you only need to configure your definition of done - the verification. How should the job verify the commits? For example compiling, running unit tests and checking that coverage does not drop.

The simple Git workflow

Get your repository up to date:

git fetch --prune

git checkout master

git pull origin master

Create a feature- or development- or... branch

git checkout -b feat_1337-improved-login-screen

...work, stage and commit changes.

Then push changes to a ready branch, which is basically just a branch following a naming conventions for branches matching ready/**.

git push origin feat_1337-improved-login-screen:ready/feat_1337-improved-login-screen

The change will be picked up by the plugin if configured as shown in next section.

  • You can then delete your local branch and continue with a new feature or development branch.
  • You are free to push to any branch name not matching the ready branch naming convention without triggering an integration build.

Integration flow

Below is a simplified diagram of what Pretested Integration actually does once you've pushed your branch to integrate.
At the start, HEAD is at the integration branch, ready to merge in your branch.
In the end the merge result is verified and, if successful, pushed.

Help and error messages

*NOT DONE YET - work in progress* (We have gathered some common errors and problems seen, together with some suggested solutions - Help and error messages)

Demo jobs and example on commit messages and output

We have created some Jenkins job on our public Jenkins master that shows the plugin in action, in the version we have installed (typically latest one or even latest snapshop).

There is actually a demo job you can run, that triggers a demonstration:

The commits end up in our Github test repository called blackhole

Accumulated strategy - commit message example

Version 2.3.0 vs. version 2.2.3

Note the difference with committer and author between the two version.
From version 2.3.0 the integration commit is authored by author of the last commit on the development branch.

In the example it is the "demo-job-user" that is the author, while the committer the git user configured for the Jenkins build slave environment (ReleasePraqma in this example). |

Accumulated strategy commit message formatting

The accumulated commit message can not be autogenerated by git, as the squashed commit message, so the plugin creates it by extracting relevant information, formatting it and creating a squash commit like message.

The formatting rules we use are:

  • indentation using space
  • date strings uses Java simple date formatter with English locale, and date format "EEE MMM d kk:mm:ss yyyy ZZZZ"
  • date string uses your local time zone, see (JENKINS-29377)
  • Quotation marks (") are replaced single pling, (') if found in any of the contributing commit messages

Squashed strategy - commit message example

Version 2.3.0 vs. version 2.2.3

Note the difference with committer and author between the two version.
From version 2.3.0 the integration commit is authored by author of the last commit on the development branch.

In the example it is the "demo-job-user" that is the author, while the committer the git user configured for the Jenkins build slave environment (ReleasePraqma in this example). |

Accumulated strategy - build console output example

Output from the plugin is prefixed with [PREINT].

[PREINT] Preparing environment using Pretested Integration Plugin 2.3.0-SNAPSHOT (cbd24)
[PREINT] Checking out integration branch master:
[PREINT]  git checkout -B master origin/master
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git checkout -B master origin/master
[PREINT]  git pull origin master
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git pull origin master
[PREINT] Preparing to merge changes in commit b1e190f90a8c1a22402af4492c70ae45eb32c981 on development branch origin/ready/feature-accumulated-32 to integration branch master
[PREINT] Collecting commit messages on development branch origin/ready/feature-accumulated-32
[PREINT] Done collecting commit messages
[PREINT] Collecting author of last commit on development branch
[PREINT] Done colecting last commit author: demo-job-user <support@praqma.net> 1435686815 +0200
[PREINT] Starting accumulated merge (no-ff) - without commit:
[PREINT]  git merge -m "Accumulated commit of the following from branch 'origin/ready/feature-accumulated-32':

commit b1e190f90a8c1a22402af4492c70ae45eb32c981
Author: demo-job-user <support@praqma.net>
Date:   Tue Jun 30 19:53:35 2015 +0200

    Demo commit message for Pretested Integration:

    This is a longer commit message about what this commit is about.

    We use this commit to illustrate the Prested Integration Plugin commit messages, based on their strategies. There are two demo jobs, one for each strategy and a job that does two commits, one on two different branches to be picked up as ready branches.


" b1e190f90a8c1a22402af4492c70ae45eb32c981 --no-ff --no-commit
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git merge -m "Accumulated commit of the following from branch 'origin/ready/feature-accumulated-32':

commit b1e190f90a8c1a22402af4492c70ae45eb32c981
Author: demo-job-user <support@praqma.net>
Date:   Tue Jun 30 19:53:35 2015 +0200

    Demo commit message for Pretested Integration:

    This is a longer commit message about what this commit is about.

    We use this commit to illustrate the Prested Integration Plugin commit messages, based on their strategies. There are two demo jobs, one for each strategy and a job that does two commits, one on two different branches to be picked up as ready branches.


" b1e190f90a8c1a22402af4492c70ae45eb32c981 --no-ff --no-commit
[PREINT] Accumulated merge done
[PREINT] Merge was successful
[PREINT] Starting to commit accumulated merge changes:
[PREINT]  git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686815 +0200"
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686815 +0200"
[PREINT] Commit of accumulated merge done
[PREINT] Commit was successful
[pretested-integration-plugin__demo-job-accumulated-strategy] $ /bin/sh -xe /tmp/hudson2956180739635715796.sh
+ echo Build and verification was successful
Build and verification was successful
[PREINT] Performing pre-verified post build steps
[PREINT] Pushing changes to integration branch:
[PREINT]  git push origin master
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git push origin master
[PREINT] Done pushing changes
[PREINT] Deleting development branch:
[PREINT]  git push origin :ready/feature-accumulated-32
[pretested-integration-plugin__demo-job-accumulated-strategy] $ git push origin :ready/feature-accumulated-32
[PREINT] Done deleting development branch

Squashed strategy - build console output example

Output from the plugin is prefixed with [PREINT].

[PREINT] Preparing environment using Pretested Integration Plugin 2.3.0-SNAPSHOT
[PREINT] Checking out integration branch master:
[PREINT]  git checkout -B master origin/master
[pretested-integration-plugin__demo-job-squashed-strategy] $ git checkout -B master origin/master
[PREINT]  git pull origin master
[pretested-integration-plugin__demo-job-squashed-strategy] $ git pull origin master
[PREINT] Preparing to merge changes in commit 26576c087d13a50179a13054cc6edac603271abd on development branch origin/ready/feature-squashed-32 to integration branch master
[PREINT] Collecting commit messages on development branch (for debug printing): origin/ready/feature-squashed-32
[PREINT] Done collecting commit messages
[PREINT] Collecting author of last commit on development branch
[PREINT] Done colecting last commit author: demo-job-user <support@praqma.net> 1435686849 +0200
[PREINT] Starting squash merge - without commit:
[PREINT]  git merge --squash origin/ready/feature-squashed-32
[pretested-integration-plugin__demo-job-squashed-strategy] $ git merge --squash origin/ready/feature-squashed-32
[PREINT] Squash merge done
[PREINT] Merge was successful
[PREINT] Starting to commit squash merge changes:
[PREINT]  git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686849 +0200"
[pretested-integration-plugin__demo-job-squashed-strategy] $ git commit --no-edit "--author=demo-job-user <support@praqma.net> 1435686849 +0200"
[PREINT] Commit of squashed merge done
[PREINT] Commit was successful
[pretested-integration-plugin__demo-job-squashed-strategy] $ /bin/sh -xe /tmp/hudson3927922213950794742.sh
+ echo Build and verification was successful
Build and verification was successful
[PREINT] Performing pre-verified post build steps
[PREINT] Pushing changes to integration branch:
[PREINT]  git push origin master
[pretested-integration-plugin__demo-job-squashed-strategy] $ git push origin master
[PREINT] Done pushing changes
[PREINT] Deleting development branch:
[PREINT]  git push origin :ready/feature-squashed-32
[pretested-integration-plugin__demo-job-squashed-strategy] $ git push origin :ready/feature-squashed-32
[PREINT] Done deleting development branch

Manual building

In general you are not able to use 'Build now' with a pretested integration job configuration. The Pretested Integration Plugin is first in action when a "workspace" is handed over from the Git Plugin (or Multiple SCMs Plugin), and typically those will serve the last build revision again. If that succeeded last time, the revision is deleted after the integration and retrying the integration fails.

Some successful and failing cases using manual builds are:

  • If last build failed, thus the integration failed, for a non-persistent error (slave problem, licensing problem ...) rebuilding the job can succeed if no other build have been executed since last time.
  • Rebuilding after a successful build will typically fail, if the build is rebuilding the same revision again. That revision doesn't not exist as it is deleted after a successful integration in last build.
  • If you have done a job configuration change, and need to trigger the job to test the configuration you typically need to make a commit that triggers the job. Push a commit to a ready branch, or wait for one.
  • There is a work around, that often enables you to build manually: Make the job parametrized with BRANCH_TO_BUILD and use that variable in the 'branch specifier'. Make BRANCH_TO_BUILD have the default ready-branch specifier, so if not given the job works as if there were no parameters. If you now build the job manually, you can type in a branch to build.

You are welcome to contribute with ideas and use-cases for manual build and support for it on the roadmap in the Trello board.

Using multiple repository configurations

The plugin is designed to work on only one repository - as in you can only integrate on one repository pr. Jenkins job, but your Jenkins job can be used with multiple repositories for eg. checking out several repositories into a given folder structure you need for building the software. Though we consider this a work-around for not having the right dependency management, it can be done. You have the choice of using the Git Plugin or the Multiple SCMs Plugin, but need to follow the configuration examples below.

Requirements if using multiple repository jobs

Every repository should be explicitly and uniquely named* in the git repository configuration under 'Advanced'. Then use this as integration repository in the prested integration configuration

Example with multiple git repositories

  • The two repositories 'capital-letters' and 'numbers' are checked out to the same workspace.
  • 'numbers' are the repository that is being integrated, so the branch specifier will typically match that patter ('capital-letters' might just be needed for building)
  • The pretested integration configuration now explicitly states that the 'numbers' repository is the integration repository.
  • Also remember to add the two additional behaviours to ensure a clean work space and that old deleted branches are cleaned

Basically the same setup needs to be applied if using the Multiple SCMs plugin, but you can benefit from additional behaviours like checking each git repository out to a subdirectory if needed. Still every git repository must be explicitly and uniquely named.

Jenkins Job DSL

Available options
job {
  wrappers {
  	pretestedIntegration(String integrationStrategy, String branch, String repository)
  }
  publishers {
  	pretestedIntegration()
  }
}
Example
job("foo_integration_job") {
  wrappers {
  	pretestedIntegration("SQUASHED","master","origin")
  }
  publishers {
  	pretestedIntegration()
  }
}

Acknowledgement

Code contributions were initially made by Computer Science students at University of Copenhagen as part of a study project

Ronni Elken Lindsgaard
Alexander Winther Uldall
Esben Skaarup
Andreas Frisch

Issues

type key summary

Data cannot be retrieved due to an unexpected error.

View these issues in JIRA

Changes

Version 2.4.1
  • Made a clearer error message when the plugin fails to push empty commits. (#15
Version 2.4.0
  • Added Credentials plugin support by replacing remaining CLI calls with GitClient implementations. (issue #29104)
  • No longer refuses to integrate branches whose names contain 'master'. (Still blocks integration of 'master'.) (issue #31138)
  • No longer marks builds with SUCCESS if the Pretested Integration step fails. (issue #30465)
  • Fixed post-install plugin description. (issue #31388)
  • Plugin version is now printed to console. (issue #31139)
Version 2.3.4
  • Single commits are fast-forward merged when possible. (issue #30891)
Version 2.3.3
Version 2.3.2
Version 2.3.1
  • Squash integration strategy no longer replaces the message of single commits. (issue #30603)
Version 2.3.0
  • More logging and information in general for better debugging and understanding why builds fail. Part of feature implementation and re-factoring for JENKINS-27690, JENKINS-28590
  • Output to build console is extended and streamlines between the two different integration strategies (JENKINS-28590)
  • JENKINS-28590: Author of the integration commit is the author of the last commit on the development branch (instead of always the build user).
  • JENKINS-28596: It is possible for the accumulated to use a custom integration branch, a bug was fixed in this area.
  • Test were written to check commit message with quotes will not fail integration ( JENKINS-27662). There are still issue though, reported in unfixed issue JENKINS-28640
  • Changed test strategy toward static git repositories, that are added as test resources for the functional tests
  • JENKINS-27697: Improvements around releasing files for test repositories, so functional tests for the plugin will also run on Windows
  • JENKINS-29369: Changed accumulated commit message date strings to be English locale. (Not fixed is that timezone still local - see JENKINS-29377 and discussion here: https://trello.com/c/aY5d8Sxd/130-jenkins-29377-date-formatting-should-be-utc-in-accumulated-msg)
  • JENKINS-28640: Quotationmarks in commit message is replaced with single plings, as using quotationmarks lead to merge failure as they were not escaped properly.
Version 2.2.3
  • JENKINS-27516 Plugin hangs - reproduced with failed earlier build. A regression problem, related to never Jenkins cores and that the plugin used semaphores. Problem was seen, by the plugin prints the version number in the job console and then the job hangs.
Version 2.2.1
  • JENKINS-26568 New accumulated commit message. Almost identical to what you get from squash commit, thus it have much more traceability by including the commit messages from all included commits.
Version 2.2.0
Version 2.1.2
  • Implemented logging tracing
Version 2.1.1
Version 2.1.0
  • Protected master branch (Plugin tries to delete origin/master JENKINS-24286)
  • Re-using last commit message in accumulated strategy (Improve commit message JENKINS-24285)
  • Removed the "origin" from the description (JENKINS-24284)
  • When squashing commits, now using author from tip of branch (JENKINS-24443)
  • Additional tests added as well
Version 2.0
  • Git integration is now supported
Version 1.1
  • Dependency of Mercurial plugin set to 1.39 due to previous failure to trigger on merge commits
  • Removed UI elements that should not have been there
Version 1.0
  • Release of the first stable version
ArchivesGet past versions
Labels