Cloudify

Provides tight integration between Jenkins and Cloudify.

The plugin integrates with popular infrastructure automation toolchains, providing a consistent way to manage all environments through a single Jenkins plugin. It includes build steps and examples that provide seamless integration with AWS CloudFormation, Azure ARM, Terraform, Ansible and Kubernetes (including GKE, EKS, and AKS), as well as OpenShift and KubeSpray.

Cloudify users can use the plugin to call Cloudify API through fine-grained build steps, saving the need to use explicit REST calls or custom scripts to refer to Cloudify Manager, greatly simplifying the use of Cloudify Manager through Jenkins jobs and pipelines.

For more information, as well as a live demo, see: https://cloudify.co/Jenkins

To download Cloudify Community Edition (for CentOS / RHEL), refer to: https://docs.cloudify.co/staging/dev/trial_getting_started/set_trial_manager/download_community/

To download Cloudify Community Edition's Docker image: https://hub.docker.com/r/cloudifyplatform/community

To learn more about Cloudify: https://cloudify.co

• Examples
• Terminology
  • Deployment Outputs File
  • Inputs Mapping File
• Installation
• Configuration
  • Cloudify Manage Endpoint
  • Credentials
• General Usage Notes
• Available Build Steps: Basic
  • Upload Cloudify Plugin
  • Upload Cloudify Blueprint
  • Delete Cloudify Blueprint
  • Build Cloudify Environment
  • Delete Cloudify Environment
  • Execute Cloudify Workflow
  • Convert Cloudify Environment Outputs/Capabilities to Inputs
• Available Build Steps: Provisioners and Orchestrators
  • Create Kubernetes Resources
  • Run Ansible Playbook
  • Create Azure ARM Deployment
  • Create CloudFormation Stack
  • Apply Terraform Module
• Cloudify's Build Wrapper

This repository contains a few examples of job definitions making use of features included in this plugin. You can find them in examples/jobs.

To import these definitions into Jenkins, use the Jenkins CLI. For example:

java -jar jenkins-cli.jar -s <jenkins_url> create-job <job_name> < path_to_job_xml_file 

For example:

java -jar jenkins-cli.jar -s <jenkins_url> create-job <job_name> < examples/jobs/cloudify-wrapper-with-git.xml 

Certain build steps (such as the "Create Environment" build step, or the "Cloudify" wrapper) allow you to write a "Deployment Outputs File" at the end of creating the environment. This file can be used by subsequent build steps, to gather information about the environment that had just been created.

The file is a JSON file, which adheres to the following format:

{
    "deployment": {
        "id": deployment_id
    },
    "outputs": {
        "output_1_name": output_1_value,
        "output_2_name": output_2_value,
        ...
    },
    "capabilities": {
        "cap_1_name": cap_1_value,
        "cap_2_name": cap_2_value,
        ...
    }
}

For example:

{
    "deployment": {
        "id": "test_env"
    },
    "outputs": {
        "endpoint": "10.0.0.131",
        "auth_info": {
            "username": "admin",
            "password": "very_secret"
        }
    },
    "capabilities": {}
}

The example above shows two outputs (one of them is a dictionary), and no capabilities.

Often, the outputs of a deployment (see "Deployment Outputs File" above) are used, in whole or in part, as inputs to subsequent Cloudify operations. This transformation can be accomplished in various Cloudify build-steps by providing an inputs mapping file, which is a YAML/JSON file that provides mapping information.

The structure of an inputs mapping file is as follows:

{
    "outputs": {
        "output_1_name": "input_1_name",
        "output_2_name": "input_2_name",
        ...
    },
    "capabilities": {
        "cap_1_name": "input_3_name",
        "cap_2_name": "input_4_name",
        ...
    }
}

For example, considering the outputs file above, and the following mapping file:

{
    "outputs": {
        "endpoint": "endpoint_ip",
        "auth_info": "user_info"
    }
}

The resultant JSON file will look like this:

{
    "endpoint_ip": "10.0.0.131",
    "user_info": {
        "username": "admin",
        "password": "very_secret"
    }
}

At the moment, this plugin is not available through Jenkins' official plugins repository. To install the plugin, download the HPI file from the "releases" section and install it via Jenkins' "Advanced" panel in the "Manage Plugins" section.

You should define the Cloudify Manager endpoint in Jenkins' console ("Manage Jenkins" -> "Configure System", look for "Cloudify Manager").

You can also specify a Cloudify tenant to use by default. If none is provided, then Cloudify's default tenant (default_tenant) will be used.

NOTE: When using an SSL-secured Cloudify Manager, the Manager's external-facing certificate must either be:

• Signed by a publicly-trusted CA; or
• Signed by a CA whose certificate exists in the JRE's trust store; or
• Imported (along with any intermediary certificates, if any) into the JRE's trust store

Most Cloudify-related functionality requires credentials to operate. Cloudify doesn't offer its own Jenkins credentials type; you should use the standard, out-of-the-box "Username with Password" credentials type.

The passing of credentials to Jenkins' Cloudify functionality is done differently depending on whether your Jenkins project is a freestyle project or a pipeline project (more commonly used nowadays).

When using a build step through a freestyle job, you need to provide the ID of a credenetials entry. Usually, you would prefer to receive this ID through a job parameter, in which case you can use Jenkins' standard parameters expansion syntax. For example: ${cfy_credentials} will return the value of a job parameter named cfy_credentials.

When using a build step through a pipeline, there are two ways to provide credentials:

• Through the credentialsId parameter, which receives the ID of a credentials entry (with similar semantics to using a step in freestyle jobs, as described above). Note that, at the moment, this method can't be used with user-scoped credentials.

• by providing the username and password parameters. This is useful when you can't use the credentialsId parameter, or when you prefer to use the traditional withCredentials syntax, which exports the username and password to variables. For example:

withCredentials([usernamePassword(credentialsId: "${params.cfy_credentials}", usernameVariable: 'CFY_USERNAME', passwordVariable: 'CFY_PASSWORD')]) {
    deleteCloudifyEnv username: "${CFY_USERNAME}",
        password: "${CFY_PASSWORD}",
        ...
        ...
}

Use this build-step to upload a plugin to Cloudify Manager.

This build-step uploads a blueprint to Cloudify Manager. The blueprint source may be provided as either:

• Path to a directory on the local filesystem
• Path to an archive on the local filesystem
• A URL to a tar.gz file

In addition, the blueprint's main YAML file must be provided.

Use this build-step to delete a blueprint from Cloudify Manager by its ID.

Use this build-step to create a Cloudify deployment.

Use this build-step to delete a Cloudify deployment.

Use this build-step to execute a workflow on a deployment.

Use this build-step to transform a Deployment Outputs File to a standard Deployment Inputs File (see "Inputs Mapping File" above).

The plugin provides build-steps to facilitate the usage of popular provisioners and orchestrators, allowing job/pipeline authors to use such tools through Cloudify in a seamless manner.

This build step receives a Kubernetes application definition, along with context parameters, and runs it using Cloudify's official Kubernetes plugin.

• Cloudify's Kubernetes Plugin installed on Cloudify Manager

The Kubernetes build step is flexible to accommodate two methods of authenticating to Kubernetes:

• Using a token
• Using GCP Service Account credentials

In order to use a token, there are two options:

• Providing the ID of a Jenkins credential entry to the "API Key Credentials ID" / "apiKeycredentialsId" parameter. The credentials entry must either be of type "string" ("Secret Text") or type "file" ("Secret File").

• Providing the path to a file containing the token. The path must be relative to the Jenkins workspace root.

In order to use a GCP Service Account, there are two options:

• Providing the ID of a Jenkins credential entry to the "GCP Credentials ID" / "gcpCredentialsId" parameter. The credentials entry must either be of type "string" ("Secret Text") or type "file" ("Secret File").

• Providing the path to a YAML/JSON file containing the service account details. The path must be relative to the Jenkins workspace root.

At a minimum, you should provide a value to the "Kubernetes Master" / "k8sMaster" parameter, pointing at the Kubernetes Master.

In addition, you can provide additional parameters to control the connection to the Kubernetes Master, such as SSL parameters.

This build step receives the path / URL of an Ansible playbook, along with other context parameters, and runs it using Cloudify's official Ansible plugin.

• Cloudify's Ansible Plugin installed on Cloudify Manager

This build step receives the path / URL of an Azure ARM template, and template parameters. The template is then deployed on Azure, using Cloudify's Azure plugin.

• Cloudify's Azure Plugin installed on Cloudify Manager

Certain parameters may be omitted, in which case the value will default to Cloudify secrets:

This build step receives the path / URL of an AWS CloudFormation template, as well as template parameters, and creates a CloudFormation Stack off it, using Cloudify's AWS plugin.

• Cloudify's AWS Plugin installed on Cloudify Manager

Certain parameters may be omitted, in which case the value will default to Cloudify secrets:

This build step receives the path / URL of a Terraform module, as well as module variables, and applies the module, using Cloudify's Terraform plugin.

• Cloudify's Terraform Plugin installed on Cloudify Manager
• Terraform installed

Certain parameters may be omitted, in which case the value will default to Cloudify secrets:

The Cloudify Plugin for Jenkins also provides a Jenkins Build Wrapper. The wrapper provides a convenience method for creating a Cloudify environment before the build, and destroying it afterwards.

The wrapper can be activated by checking the "Cloudify Environment" box under "Build Environment" during job definition.