CI Tools and Best Practices in the Cloud

Continuous Integration

Subscribe to Continuous Integration: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Continuous Integration: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Continuous Integration Authors: Pat Romanski, Liz McMillan, Elizabeth White, Amit Gupta, XebiaLabs Blog

Related Topics: Software Testing Journal, Continuous Integration, DevOps Journal

Blog Feed Post

Measuring #DevOps Success | @DevOpsSummit #CD #DX #Agile #Serverless

When and where is activity occurring in my delivery pipeline?

Measuring DevOps Success in Your Software Delivery Pipeline
By Jason Lenny

When it comes to measuring the success of your DevOps rollout, it can be challenging to identify the right metrics that will provide intelligence while avoiding the trap of vanity metrics that indicate action—but not necessarily progress—towards the outcome you’re looking for.

In my experience, the most valuable metric of all is the lead time between when you make a commit in source control and when that change makes it to your consumers. Some very mature organizations have even been able to link this metric to validated learning or planned outcomes in production (i.e., user engagement, revenue, or even a pivot decision). This sort of full-cycle measurement closes the DevOps build-measure-learn loop and gives you unparalleled insight into the performance of your overall delivery metrics.

This sort of advanced metric, however, can be very difficult to implement; it often requires process, technology, and cultural changes within your team to be successful. If your organization is just beginning its DevOps journey, it can be unclear where you can start getting value immediately. For teams at that point, I have listed a few ideas here that are other good areas to begin.

When and where is activity occurring in my delivery pipeline?
Measuring when individual parts of your delivery pipeline are heavily utilized can offer surprising insights on when teams may be overloaded. If your source control, build, and testing systems remain idle until late in your sprints then suddenly become overwhelmed with changes, it would suggest changes to your planning or software development process could yield real benefits. For example, implementing something like Kanban into your development process could smooth the flow of work into the system and resolve what teams may be experiencing as late-sprint chaos. Within XL Release, this sort of metric can be clearly visible within the longest task/phase reports.

What kind of activities across systems are failing and how frequently?
The reliability of the various technical processes within your pipeline can have an outsized effect on the flow of changes through your system—not to mention team morale. Unreliable tests, flaky build systems, or other components of your pipeline that are unreliable will quickly form a bottleneck to teams looking to deliver more quickly and independently. Keep a close eye on these to ensure you aren’t tripping over your own feet.

Longest releases-phases-tasks views in XL Release provide DevOps intelligence

The longest releases/phases/tasks views within XL Release can show you the way to inefficient activities.

How long does it take to get value into a consumer’s hands?
This is the cycle-time metric I mentioned earlier. Even if you aren’t yet ready to implement the full build-measure-learn loop, you are probably still ready to break your pipeline down into interesting pieces. For example, you can measure the duration and activities within your development, testing, and release phases individually to gain insight on where within these processes you have opportunity to improve. Your delivery teams probably already have insight into where the least efficient parts of your pipeline are. Begin by modeling these and you’ll be surprised what you learn.

Are we releasing to customers more frequently?
Another core DevOps metric is how frequently you release to your customers. A very important contributor to lead time, releasing more frequently will bring down the number of changes in each release, resulting in more frequent but less risky change.

XL Release Releases Per Month Report provides DevOps intelligence

The Releases per Month report in XL Release offers a window into how you’re performing.

Are we taking less time to perform these releases?
This metric goes hand in hand with the frequency of release metric above. As you release more frequently, it is important to monitor the overall level of effort and duration.

XL Release average release duration and automation report provides DevOps intelligence

XL Release can also offer insights into duration through the Average Release Duration/Automation report.

Have you identified any other key metrics that are important for teams to track, or do you have any thoughts on the ones we’ve shared above? Please share your insights in the comments section below.

The post Measuring DevOps Success in Your Software Delivery Pipeline appeared first on XebiaLabs.

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.