Minimum Viable Improvement Step for Implementing a Continuous Delivery Pipeline

Building a Continuous Delivery Pipeline Using Jenkins

by Iris Chubb, on Apr 26, 2021 12:30:00 PM

Continuous Delivery isn't easy to get right, and a lot of organizations have struggled to try to build Continuous Delivery pipelines with Jenkins. That's because Continuous Delivery is a lot more complicated than simply using the Jenkins Pipelines feature, and if you start there, you'll most certainly fail, too.

But it's not impossible, and in this article, I'll explain how to implement Continuous Delivery Pipelines properly and how to do Continuous Delivery with Jenkins.

What is Continuous Delivery (CD)?

Continuous Delivery (CD) refers to the set of practices that Development and Operations teams use to produce and deliver software for business teams. These practices are not uniformly defined by a standards body such as ISO but instead are explored, implemented, integrated, and continuously refined as part of an organization's core business processes.

Although specific Continuous Delivery evolves differently within each organization and for every application, all organizations tend to experience the same benefits:

• Better quality of software by having fewer defects make it to production
• Faster delivery of business ideas to market by enabling faster release cycles that allow software to be changed in days or weeks rather than months or quarters
• Cheaper implementation across the entire lifecycle, including less time spent coding, deploying, and testing software changes

What is a Continuous Delivery Pipeline?

Definition of Continuous Delivery

One of the key components of Continuous Delivery is the deployment pipeline. Pipelines model the software release process by defining the servers and environments that builds will be deployed to, as well as the manual and automatic approvals required at each stage of the process.

Different applications may use different pipelines or the same application may use different pipelines for different releases. It's highly contextual.

Stages of a Continuous Delivery Pipeline

There is no "universal" CD pipeline. Every organization has different requirements, and each application within each organization will be different.

For example, a basic web application might use a pipeline with only two stages (testing and production), and simply deploy to a different folder on the same server.

Another application may require a dozen stages, each with multiple targets that go to different environments, and all sorts of automatic and human approvals to meet compliance requirements.

Stages in your CD pipeline should model the release process that the business is already using, not a technical process that engineers want to use.

No matter what business structure or value you seek, CD pipelines require an input (i.e., the build to deploy), and that input comes from a Continuous Integration (CI) process.

This input is a build artifact, essentially a zip file containing the code that will be deployed to each server across each environment. When you combine build artifacts created from a CI process and delivery pipelines together, you have the essence of Continuous Delivery: automated, end-to-end software delivery with human judgment.

Bottom line, CD processes must fit organizational and business needs—not the other way around.

Why is Continuous Delivery Important?

The true value of CI/CD is that organizations, regardless of industry, vertical, geography, market, etc., can adapt to market changes and innovate just as fast (if not faster) as their competitors.

This is accomplished by changing the software that powers the business and then releasing those applications as fast as developers can write new code and testers can verify it.

Prior to CI/CD, software releases are often held up by the unpredictable nature of developing complex changes, bugs/defects during testing, poor code quality, and by processes originally designed for ancient, mainframe-based software systems.

Continuous Delivery pipelines improve this process from start to finish. Despite being released at a higher velocity, applications can also be built and deployed to meet business needs with greater ease than in the past.

How Does Jenkins Help with Continuous Delivery?

Jenkins is one of the most popular DevOps tools. It was originally designed with one purpose in mind: to be a great build automation server. It's open-source, built for developers, and has lots and lots of plugins that you can configure to build anything.

Of course, since a build script could just be an arbitrary script, a Jenkins project could be used to do anything. And thus, Jenkins could be used for anything… even to run a CD pipeline or even PowerShell scripts.

But Jenkins brings serious limitations to businesses. Just as an all-purpose job scheduler would be a poor choice for building automation jobs, Jenkins isn't natively equipped to be an end-to-end CI/CD tool because it was never designed as one.

What is a Jenkins Pipeline?

A Jenkins Pipeline is not a Continuous Delivery pipeline.

A Jenkins Pipeline is actually a special type of project, implemented through the Pipelines plugin (originally called workflows). This plugin now comes pre-installed on Jenkins.

While they're a much more complicated type of Jenkins project, Pipelines make Jenkins project much more powerful. Based in Jenkin's Groovy-based DSL, Pipelines deliver Steps, which perform a basic task on a node. There are hundreds of available Steps, including:

• Git - checks out code from your git repository
• dotnetBuild - builds your .NET project
• JUnit - runs JUnit tests against your Java Project

Steps are organized into Stages to improve the user experience, breaking the logs down into Stages and Steps for easier navigation. Each Stage is shown as a square, so you can more quickly see where a failure occurred.

Compared with only having a single log file that has your script's console output, stages help visualize long and complex automation processes. You can also restart a job from a Stage in the script.

A Jenkins Pipeline Stage is not a Continuous Delivery stage

Jenkins Pipeline Stages are parts of a detailed, technical process that helps engineers break down a Jenkins Pipeline script and include Source Control, Build Code, Run Unit Tests, etc.  A Continuous Delivery stage, on the other hand, is part of the release process that the business is already using.

How Do You Build a Continuous Delivery Pipeline Using Jenkins?

The short answer: you can't. Jenkins on its own is a poor choice for Continuous Delivery for many reasons.

It's true that there are ways to Jenkinstein a pseudo-CD pipeline using different Jenkins plugins, like Jenkins Pipeline Input Step. Just as a PowerShell or Bash script can prompt a user for input in the middle of running, a Jenkins Pipeline script can do the same.

But remember, Continuous Delivery processes must fit organizational and business needs—and an input prompt in the middle of a Jenkins Pipeline isn't going to cut it. CD is not only automation but rigorous approvals. This minimally-viable option isn't going to meet the needs of most organizations.

Do Continuous Delivery with Jenkins

The first step: Use a dedicated Continuous Delivery tool with Jenkins. I think our product (BuildMaster) is a great choice, but I'm a bit biased.

When you use BuildMaster (or any other CD or deployment tool) with Jenkins, you get all of the value that Jenkins is currently bringing you (build automation/CI, testing, and/or running arbitrary scripts) plus a flexible release process that the business wants to use. This also has the added bonus of being more accessible to team members who aren't Jenkins experts.

Store Release Artifacts outside of Jenkins

Jenkins was not designed as a storage tool for your software assets. While you can capture build artifacts as part of a Jenkins project, these can quickly get lost in the shuffle of hundreds of CI builds across hundreds of projects.

Storing release artifacts outside of Jenkins will not only make them easier to find, but you also don't have to worry about keeping your Jenkins server tidy and purging build records. There are three general approaches to this:

• Store release artifacts in your dedicated CD tool. This way, they can be used throughout the continuous delivery process, and it'll be easy to roll back to old versions at any time.
• Upload to a Maven server. This is becoming less and less popular, as Maven was designed ages ago for Java applications.
• Publish as packages. This is a relatively new approach and involves using a Package Manager like ProGet, which gives you access to things like Semantic Versioning, and allows you to "install" them on servers, so you can see which package is installed where.

Any of these options are better than storing your build artifacts in Jenkins. That leads to all sorts of problems, including slowing down your Jenkins server (and a slower CI server wastes time and increases the risk of failed builds). More reliable, less fuss.

Jenkins is a popular tool for a reason. But for all of its strengths, it's not a good choice for building a Continuous Delivery pipeline. CD requires a tool specifically designed to meet the need for high-velocity application releases and that doesn't compromise on visibility into the full impact of changes.

cowanwoespen.blogspot.com

Source: https://blog.inedo.com/jenkins/continuous-delivery-pipeline

Share this post