CI Tools and Best Practices in the Cloud

Continuous Integration

Subscribe to Continuous Integration: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Continuous Integration: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Continuous Integration Authors: Stackify Blog, Aruna Ravichandran, Plutora Blog, Dalibor Siroky, PagerDuty Blog

Related Topics: Continuous Integration, DevOps Journal

Blog Feed Post

How We Use XL TestView By @PaulvanderEnde | @DevOpsSummit #APM #DevOps

Let me first explain how we manage the configuration of our Continuous Integration systems

How We Use XL TestView to Improve the Test Quality of XL Deploy
By Paul van der Ende

Here at the XebiaLabs we care a lot about quality. That means we have to do a lot of testing for our products. Since we have a relatively small development team, and a code base that grows rapidly, that means we have to automate our tests. To get an impression, for XL Deploy, we have automated tests for components like the user interface and the server. But we also have automated tests for all our plugins, in combination with different types of middleware, operating system, etc. This all adds up to an enormous test landscape with an Continuous Integration system containing a couple of hundreds of Jenkins jobs, and thousands of unit and integration tests.

But even if you have all of your tests automated, you still have another challenge. How are you going to monitor all those thousands of tests and track all the test results? We do a lot of integration testing with virtual servers and real middleware. Since there are so many moving parts, it’s a matter of time before you run into stability problems. With just a Continuous Integration system, it’s hard to analyse which builds/tests are flaky, which tests fail most often, where the biggest problems are so where to start improving. Continuous Integration systems generally lack the possibility to analyse your test results.

Well, it seems we have a product at XebiaLabs to solve this problem: XL TestView. At XebiaLabs we regularly have Innovation Days where developers try to tackle problems we are facing. This time we took the opportunity to integrate the Continuous Integration system of XL Deploy with XL TestView, to get more insight into our test results.

Before I explain how we did that, let me first explain how we manage the configuration of our Continuous Integration systems. Like I already mentioned, we have a lot of Jenkins jobs. How do we organise and administrate them? Some time ago we have scripted our entire Jenkins infrastructure configuration with Jenkins job dsl. This allows us to automatically generate all job configuration with the push of a button.

Since all the information about our test infrastructure is also in this system, we decided to extended the Jenkins job DSL to make it possible to create test specifications in XL TestView using the REST api. When we create a Jenkins job, we also create a new test specification in XL TestView. So now when we automatically configure Jenkins, we also automatically configure XL TestView for XL Deploy, and directly configure all Jenkins jobs to upload the test results to XL TestView using the Jenkins plugin for XL TestView. This plugin collects all the test results from the Jenkins workspace and uploads them. All this made it possible to track all our thousands of tests in XL TestView, in just 1 day.

And this is what that looks like:

Jenkins job configuration to upload the test results to XL TestView:

testview1

A big list of test specifications in XL TestView

testview2

XL TestView dashboard showing the first results:

testview3

Learn more and try it for yourself! Visit https://xebialabs.com/products/xl-testview/ and get started!

The post How we use XL TestView to improve the Test Quality of XL Deploy appeared first on XebiaLabs.

Read the original blog entry...

More Stories By XebiaLabs Blog

XebiaLabs is the technology leader for automation software for DevOps and Continuous Delivery. It focuses on helping companies accelerate the delivery of new software in the most efficient manner. Its products are simple to use, quick to implement, and provide robust enterprise technology.