Amazon EC21.37Minimum Jenkins requirement: 1.651.3ID: ec2
Allow Jenkins to start slaves on EC2 or Eucalyptus on demand, and kill them as they get unused.With this plugin, if Jenkins notices that your build cluster is overloaded, it'll start instances using the EC2 API and automatically connect them as Jenkins slaves. When the load goes down, excessive EC2 instances will be terminated. This set up allows you to maintain a small in-house cluster, then spill the spiky build/test loads into EC2 or another EC2 compatible cloud.
First, go to EC2 and sign up for the service. Once you've installed the plugin, you navigate to the main "Manage Jenkins" > "Configure System" page, and scroll down near the bottom to the "Cloud" section. There, you click the "Add a new cloud" button, and select the "Amazon EC2" option. This will display the UI for configuring the EC2 plugin. Then enter the Access Key and Secret Access Key which act like a username/password (see IAM section). Because of the way EC2 works, you also need to have an RSA private key that the cloud has the other half for, to permit sshing into the instances that are started. If you have already been using EC2 and have your own key, you can paste it here. Otherwise, you can have Jenkins generate one. If you let Jenkins generate one, you should save this private key in your file system as well, as you'll need this to interactively logon to EC2 instances.
Once you have put in your Access Key and Secret Access Key, select a region for the cloud (not shown in screenshot). You may define only one cloud for each region, and the regions offered in the UI will show only the regions that you don't already have clouds defined for them.
Use "Test Connection" button to verify that Jenkins can successfully talk to EC2. If you are using UEC you need to click on Advanced and fill out the endpoint details for your cluster.
Next, configure AMIs that you want to launch. For this, you need to find the AMI IDs for the OS of your choice. ElasticFox is a good tool for doing that, but there are a number of other ways to do it. Jenkins can work with any Unix AMIs. If using an Ubuntu EC2 or UEC AMI you need to fill out the rootCommandPrefix and remoteAdmin fields under 'advanced'. Windows is currently unsupported.
Configuring labels allow Jenkins to pick the right AMI to start. For example, if all your existing slaves labeled "solaris" are fully busy and you have more builds that are tied to the "solaris" label, Jenkins will start the AMIs that have the "solaris" label.
Init script is the shell script to be run on the newly launched EC2 instance, before Jenkins starts launching a slave agent. If the AMI doesn't have Java pre-installed, you can do this in the init script. This is also a good place to install additional packages that you need for your builds and tests. The init script is located at /tmp/init.sh and is owned and run by the user account specified in the "Remote User" field (so use of "sudo" may be required for non-root accounts).
Configure several AMIs if you need different OS images.
With this setting, your Jenkins will automatically start instances when the load goes up, and the instances will be terminated (or stopped - see below) automatically if it's idle for more than 30 mins.
By default, instances are terminated when the idle timeout period expires. You can change this by specifying the Stop/Disconnect on Idle Timeout flat in the Advanced properties of the AMI configuration. If this is specified, the instance is stopped when the timeout expires. If the instance is required again, then the plugin will look for a stopped instance that exactly corresponds to the AMI specification and resume it if found. Otherwise a new instance is started. Having the instances be stopped instead of terminated is useful when you are using EBS volumes and want to keep them mounted for the life of the instance and reuse the instance for long periods of time. This can greatly reduce the startup time of the instance since it does not have to build the volume from the snapshot.
Spot instances are similar to EC2 instances, except for a few key differences. They are generally more affordable, but they have the possibility of being terminated if someone has bid more on them than you have and can take longer to provision. To mitigate some of these issues, your Jenkins and Slave AMIs will need a bit of configuration to convert the Spot slaves to register with Jenkins when they spawn up. Due to these changes, it may appear that a Spot slave will have failed (with a red X), but the message associated with the error will inform you if the Spot instance just has not called back yet or not. For more information on Spot instances, read the information here: http://aws.amazon.com/ec2/spot-instances/
Enable Spot Request
To enable to use of Spot as opposed to EC2 for an instance, the "Use Spot Instance" check box must be checked. Also, a bid price must be specified. If you want to determine what the current price of the instance is without going to the AWS website, pick your region and instance type(as you already should) and click "Check Current Spot Price".
To choose between a Persistent or One Time bid (to keep the bid alive until cancelled or to stop the bid after it has been fulfilled once), choose from the drop down menu.
Configure Jenkins for Spot Support
For Jenkins, the major configuration change will be if you have a restrictive firewall, since these instances need to connect back to Jenkins. The first configuration change to Jenkins is to change your Jenkins URL option in the Configure Jenkins page to be the external URL to the server. One port that needs to be open is the one that you use to access Jenkins, the other is the JNLP port, which is generally randomly assigned. To set the JNLP port to something predictable, follow the Connection Mechanism section on this page. Jenkins CLI
Configure AMI for Spot Support
In order for your EC2 instance to know that it is to be a Jenkins slave, it must be preconfigured with start up commands so that it can register itself with Jenkins. The Jenkins information is passed to the Spot slaves via EC2 user-data. This information includes the name that Jenkins has given the slave, and the configured URL for the Jenkins master node.
Sample scripts for assisting in configuring an Ubuntu based AMI to work with the Jenkins ec2-plugin and Spot slaves are included with the installation of the plugin.
The config script is run by the user from the EC2 instance with root access. It installs Java onto the instance and fetches the startup script and sets it to run when the machine starts up. It can be retrieved from the above url using a command like wget, or copied over using a tool like scp.
Once the scripts have been downloaded, the script can be run. Running this script will also run the ubuntu-init.py script, so there is no need to run it on its own.
Note: ‘http://” will be prepended to the jenkins_server parameter
The config script then fetches the startup script and sets up the AMI to register itself with a Jenkins master node when it gets started.
After setting up the image, you can save the image using Amazon’s EC2 web console. To do this, right click on your instance from the console and select “Create Image (EBS AMI)”.
In order to set up additional images using other operating systems, you can configure your own startup script based on the startup script above. This script should perform the following actions when the machine is started up:
It's possible to create a separate account for jenkins using the Amazon IAM feature. Go to the IAM tab in AWS console and create a user. Then go to user's Permissions tab and use this policy (change username if your user is not called jenkins):
If you want to launch slaves with IAM Instance Profile, "iam:PassRole" permission is required.
Issues with Authentication Timeout
If you have issues with authentication timing out as a node is starting (see the stack traces in JENKINS-30284) you can use these System Properties to adjust the number of tries and timeout values:
Version 1.36 (Oct 2, 2016)
- JENKINS-38481 AWS method change causes too many slaves to be launched
Version 1.33 (May 9, 2016)
- JENKINS-34667 Provision attempt is made when possible slaves count is zero (backed out)
Version 1.32 (May 8, 2016)
Warning: Please use 1.33 instead, it has a critical fix.
- Welcome Johnny Shields as co-maintainer!
- JENKINS-26371 Ensure instance initiated shutdown behavior is consistent with stopOnTerminate flag
- JENKINS-27529 Poll for spot instances instead of JNLP launcher
- JENKINS-32588 Unable to launch linux slaves using ec2 plugin using Eucalyptus
- JENKINS-32584 ec2-plugin counts every instance (not just Jenkins slave instances) when counting slaves; incorrectly hits slave cap
- JENKINS-32690 Make manually provision really work
- JENKINS-32915 Spot instance launched one after another until capacity reached for single task in queue
- JENKINS-33945 can't provision new nodes when a matching node is marked offline
- JENKINS-34667 Provision attempt is made when possible slaves count is zero
- EC2UnixLauncher can survive Connection IOExceptions
- Bootstrap retries with fresh connection
- Many pmd/squid fixes
- Remove AWS fault instance
- Increase minimal supported version to 1.625.1 (to enable JDK7)
- Use AWS credentials to manage the IAM access key
- Applied logic to create block device mappings for both ondemand and spot slaves
- With slaves now being fully controlled by Jenkins master, there is no longer a need to be able to set "persistent" spot bids
Version 1.31 (Jan 25, 2016)
- JENKINS-32584 ec2-plugin counts every instance (not just Jenkins slave instances) when counting slaves; incorrectly hits slave cap
Version 1.30 (Jan 23, 2016)
- Add config to prefer the public IP to private IP when ssh-ing into slave
- Added common method to compute tag value and also created constants for demand and spot
- JENKINS-27601 instance caps incorrectly calculated
- JENKINS-23787 EC2-plugin not spooling up stopped nodes
- Depend on the aws-java-sdk plugin to limit AWS SDK duplication
- Upgrade AWS SDK to 1.10.26
- Terminate instance even if ec2 node deletion failed
- JENKINS-27260 SPNEGO for Windows in the EC2 Plugin
- JENKINS-26493 Use new EC2 API endpoint hostnames
- JCIFS first tries to resolve a dfs path would timeout causing a long startup delay
- JENKINS-28754 Jenkins EC2 Plugin should show timestamp in slave logs
- JENKINS-30284 EC2 plugin too aggressive in timing in contacting new AWS instance over SSH
- Use AWS4SignerType instead of QueryStringSignerType
- Add minimum timeout for windows launching
- Better exception handling in uptime check
- JENKINS-29851 Global instance cap not calculated for spot instances
- JENKINS-32439 JENKINS-32439 Incorrect slave template (AMI) found when launching slave
- Improve logging to be less verbose
Version 1.29 (Aug 2, 2015)
- Modify sed command for Ubuntu to ignore comments
- Add option to launch EBS optimized instances
- Fix to not hide windows connection errors.
- Fix confusing log message in windows launcher
- Update AWS SDK to 1.10.0 (new instance types)
- JENKINS-28268 - Allow ability to programmatically update AMI using Groovy
Version 1.28 (Jun 7, 2015)
- JENKINS-26854 - Fixing 'RequestExpired'
- Fixing instance profile credentials with 'Check AMI' and 'Check Current Spot Price' buttons
- Masking BouncyCastle for when the system version is incompatible with the required version
- Fix java download/install
- Check more places where passing a null instanceID can do a search of random spot instances
- WiP - Launch slave agent via ssh client process
- JENKINS-24359 - Overcoming limit of one cloud per region.
- JENKINS-27260 - SPNEGO for Windows in EC2 Plugin
Version 1.27 (Mar 30, 2015)
Version 1.26 (Jan 27, 2015)
- Problem starting with empty tmpdir (JENKINS-26531)
Version 1.25 (Jan 20, 2015)
- Make temporary directories configurable (JENKINS-26232)
- Add immediate check for useInstanceProfileForCredentials option
- Include new parameter in jelly's with-clauses, respect it in doFillRegions
- Fixing zombie workers
- Add option to obtain EC2 credentials from instance meta data service
- Retry updating remote tags also for on-demand instances
- Support cloud-formation compatible user-data in spot-instance slaves
- Handle AssociatePublicIp in spot instance slaves
- Consider only slaves for the requested label when decrementing excess Workload by number of pending spot instance slaves
- Fix the stopping/pending message when launching a slave
- Fix bug that caused infinite loop of NullPointerExceptions during provisioning of windows instances.
- Fix connection to Windows slaves outside us-east-1. (JENKINS-19943)
- Give the user a readable error message when testing the connection instead of stack trace if their credentials are incorrect. (JENKINS-24676)
- EC2 plugin incorrectly reports current instance count (JENKINS-19845)
- fix EC2SpotSlaves from fetching tags from a completely different instance
Version 1.24 (Aug 5, 2014)
Version 1.23 (Jun 30, 2014)
- Fixed critical problems in 1.22.
Version 1.22 (Jun 27, 2014)
- This was a bad release, not recommended. Use 1.23 instead.
Version 1.21 (Feb 28, 2014)
Version 1.20 (Nov 8, 2013)
- Use the Node Iterator API Plugin to track provisioned nodes.
Version 1.19 (Oct 8, 2013)
- Spot instance support (pull request ec2-plugin/43)
- Fixed a problem with multiple EC2 Clouds (JENKINS-15081)
- Amazon IAM support (JENKINS-17086)
- Fixed Eucalyptus connectivity support (JENKINS-18854)
- Fix: spot instance feature cannot be used within VPC (JENKINS-19301)
- Remove slave from Jenkins even when instance termination fails (JENKINS-19500)
Version 1.18 (April 9, 2013)
- Add m3.xlarge and m3.2xlarge instance types
- Failure starting slave nodes (JENKINS-15319)
- Tags feature is broken (JENKINS-15239)
- EC2 Nodes which share an AMI ID get the wrong labels (JENKINS-7690)
- Sometimes starts the wrong instance (JENKINS-15158)
- Stopped (as opposed to terminated) slaves are counted against the active instance count for the purpose of launching; can prevent launching of instances (JENKINS-7883)
- Upgrade aws-java-sdk dependency to 1.3.30
- Explicitly add MIT license to all plugin code
- Fallback a manual or timeout-based terminate to stop if terminate fails (to avoid charges)
- Give Jenkins nodes useful names (JENKINS-15078)
- Keep track of instances being provisioned; use this count when determining total/AMI instance caps (JENKINS-6691)
- Bring back remoteFS in the slave configuration page
- Let user configure node.mode for EC2 slaves
Version 1.17 (September 12, 2012)
- Resume stopped EC2 instances (JENKINS-14884)
- Added support for EC2 tags, VPC subnets/security groups
- Added support for public/private DNS
- EC2 documentation for VPC/security groups not clear (JENKINS-15149)
Version 1.16 (May 26, 2012)
- EC2 Userdata not being Base64 encoded (JENKINS-13897)
Version 1.15 (May 21, 2012)
- Stopped (as opposed to terminated) slaves are counted against the active instance count for the purpose of launching; can prevent launching of instance (JENKINS-7883)
- Clarification and updating of help (JENKINS-12789)
- The init script was called each time instance was connected to (JENKINS-12771)
- EC2 slaves fail to launch when using versions prior to 1.9 (JENKINS-7219)
- Force registration of converter (JENKINS-10118)
- Convert to Amazon EC2 libraries(JENKINS-12539)
- Allow non-root user name (JENKINS-5867)
- Allow specification of security group (JENKINS-8617)
- Add support for M1 Medium instance (JENKINS-13432)
- Allow instances to be stopped (instead of terminated (JENKINS-12772)
- Option to set zone as well as region for instance (JENKINS-8946)
Version 1.14 (Feb 22, 2012)
- Fixed a typo in Tokyo region name, and added Oregon and Sao Paulo regions.
Version 1.13 (Jul 29, 2011)
- Fixed NPE (JENKINS-10467)
Version 1.12 (Jul 19, 2011)
- Label expressions are handled correctly (JENKINS-9773)
- Fixed a false-positive "check AMI" validation error (JENKINS-9415)
Version 1.11 (Mar 15, 2011)
- Fixed a bug in the form validation (JENKINS-6063)
- Reuse the client for better resource usage (pull request).
- SSH port is now configurable
- Eucalyptus doesn't report the IP address right away, so we need to keep checking. (JENKINS-5851)
- Added new Tokyo region.
Version 1.10 (Nov 7 2010)
- Added APAC region support.
Version 1.9 (Aug 11 2010)
- Fixed bug in backwards-compatibility of persisted configuration (JENKINS-6782)
Version 1.8 (Apr 09 2010)
- Fixed the fatal problem in configuration persistence (JENKINS-6113)
Version 1.7 (Mar 17 2010)
- Supported different AWS regions (JENKINS-4796)
Version 1.6 (Feb 15 2010)
Version 1.5 (Nov 20 2009)
- Number of executors wasn't properly persisted (JENKINS-4906)
Version 1.4 (Aug 5 2009)
- Jenkins does several retries on SSH logins to give sshd extra time to initialize itself (JENKINS-4119)
Version 1.3 (July 27 2009)
- User-data can be now specified to the launching instances (JENKINS-4115)
- # of executors can be now configured (JENKINS-4116)
Version 1.2 (Jun 18 2009)
- Fixed the dependency issue with recent Jenkins
Version 1.1 (May 28 2009)
- Re-implemented the instance cap so that it uses AWS API for # of instances to prevent run-away EC2 instances.
- Improved the performance of instance launches
- Fixed a bug where a long init script may cause Jenkins to start additional instances.
- If the init script has run once, don't run it again when reconnecting.
- Initial release