CI Tools and Best Practices in the Cloud

Continuous Integration

Subscribe to Continuous Integration: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Continuous Integration: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Continuous Integration Authors: Elizabeth White, Flint Brenton, Liz McMillan, Pat Romanski, Amit Gupta

Related Topics: Continuous Integration

Blog Feed Post

New Article: DevOps, Continuous Testing, and Service Virtualization

ServiceVirtualizationContinuousTestingDevOpsBob Aiello, Technical Editor of CM Crossroads, recently wrote an article explaining why organizations embracing DevOps and Continuous Delivery can't afford to move forward without Continuous Testing as well as Service Virtualization, which makes Continuous Testing possible.  

The following is an excerpt from the complete article...


DevOps provides a set of principles and practices which enable development and operations teams to communicate and collaborate more effectively. High performance teams which embrace DevOps have been demonstrating that they can deliver quality systems with amazing speed and agility. Many DevOps teams focus on scripting the application build, package and deployment process, resulting in an automated framework, known as the deployment pipeline. Deployment automation enables the organization to enjoy the benefits of continuous integration and continuous delivery, significantly enhancing productivity and agility.

Nevertheless, improving any process—from software development to manufacturing—inevitably involves an apparent trade-off between speed and quality. Understanding opportunities for acceleration is key for process improvement. Yet, speeding up the process without understanding the impact to quality opens up a distinct risk. In the same vein, organizations that are looking to accelerate their software delivery cycle need to ensure that the DevOps initiative is balanced with achieving the organization's definition of quality.

Software organizations are learning that QA and testing practices must be utilized throughout the entire software development lifecycle. Robust testing practices enable the organization to meet the demands and challenges of today’s fast-paced technology driven world. Without them, your entire team will be mired in firefighting software glitches and trying to patch bad software. The good news is that there are methodologies and tools available today that can empower you and your team to implement effective testing strategies that meet the demands of even the most complex IT systems embracing DevOps.

When an organization is looking at "Continuous Everything," an emerging best practice known as Continuous Testing is a critical component in the overall process. Another emerging best practice—Service Virtualization—enables Continuous Testing by providing anytime, anywhere access to a complete simulated test environment.

Continuous Testing

Delivering large scale application systems can be a challenging endeavor. The pressures of meeting time-to-market demands and testing ever more complex systems make it increasingly difficult to deliver reliable systems in a timely manner.

The only possible way to meet the demands of today’s business environment is to start testing from the very beginning of the software and systems development process. This approach should include testing requirements and designing systems to be testable through automated procedures.

This is where Development Testing practices are becoming increasingly popular within development organizations. Although many Development Testing techniques are longstanding industry best practices, few teams have been applying them religiously. This could very well change as more and more organizations recognize the escalating risk associated with software failure.

At the same time, QA testing is often limited by access to complete (fully-functional) build of the application under test as well as an environment that includes all of the dependencies required to execute end-to-end tests.

Developing an Effective End-to-End Test Suite

Today, the underlying architecture associated with most enterprise applications transacts primarily on a machine-to-machine basis over myriad communication protocols and structured message payloads. Ultimately, more and more leading organizations are finding that a mature, repeatable process for hardening GUI-less back-end services from the API level is now a "must have" for ensuring the security, reliability, and performance of critical transactions.

With APIs, testing a broad range of conditions and corner cases is critical, so automation comes to the forefront. The creation and execution of simple automated tests with limited or manual validation might have sufficed for internal given web services that were used internally (e.g., via SOA), but more sophisticated and extensive automation is required to be confident that APIs are robust enough to meet business expectations. You need a level of automation that gives you a comprehensive set of functional test cases that can be repeated in a systematic manner.

Recommended capabilities for this goal include an intuitive interface for automating complex scenarios across the messaging layer, ESBs, databases, and mainframes:

  • Defining automated test scenarios across the broad range of protocols and message types used in APIs: REST, WADL, JSON, MQ, JMS, EDI, fixed-length messages, etc.

  • Automating rich multilayer validation across multiple endpoints involved in end-to-end test scenarios.

  • Parameterizingtest messages, validations, and configurations from data sources, values extracted from test scenarios, or variables.

  • Defining sophisticated test flow logic without requiring scripting.

  • Visualizing how messages and events flow through distributed architectures as tests execute.

The Test Environment Access Needed to Test Continuously

With the convergent trends of Agile/parallel development and increasing system complexity/interdependency, it has become extremely rare for a team to have ubiquitous access to all of the dependent applications required to execute the type of complete end-to-end tests outlined above. By leveraging Service Virtualization to remove these constraints, an organization can gain full access to (and control over) the test environment—enabling Continuous Testing to occur as early and often as needed.

Service virtualization enables rapid iterative development by providing simulated test environments that can help scale continuous testing. The goal of service virtualization is to simulate interfaces and resources that may not always be available for testing due to cost or other real world constraints. This emerging industry best practice promises to provide a much more robust and comprehensive approach to ensuring that we can continuously deliver defect-free code.

Want to start testing the component you just built even though not much else is completed? Don't have 24/7 access to all the dependencies involved in your testing efforts—with all the configurations you need to feel confident that your test results are truly predictive of real-world behavior? Tired of delaying performance testing because access to a realistic environment is too limited (or too expensive)?

Service Virtualization can remove all these constraints.

Service Virtualization: What's Involved?

Software components and system interfaces may be simulated to allow the applications to be tested against virtualized test assets that look and act as if they are the actual real assets, which are often few in number and only available at specific times. Virtualized test assets look and act like the real thing but may be duplicated and available at times when the real assets are not available to the testing team, possibly because they are being used to run production. Virtualizing your test assets enables your organization to create robust test frameworks that can provide comprehensive test coverage while keeping costs low...


You can read the complete DevOps, Continuous Testing and Service Virtualization article at Configuration Management Best Practices.


DevOpsContinuousDeliveryNew Paper: Continuous Testing for DevOps

Want to learn more about why and how Continuous Testing’s real-time objective assessment of an application’s business risks is a critical component of DevOps CALMS (Culture, Automation, Lean, Metrics, Sharing)?

Read the new "Continuous Testing for DevOps: Evolving Beyond Automation" white paper.

Read the original blog entry...

More Stories By Wayne Ariola

Wayne Ariola is Vice President of Strategy and Corporate Development at Parasoft, a leading provider of integrated software development management, quality lifecycle management, and dev/test environment management solutions. He leverages customer input and fosters partnerships with industry leaders to ensure that Parasoft solutions continuously evolve to support the ever-changing complexities of real-world business processes and systems. Ariola has more than 15 years of strategic consulting experience within the technology and software development industries. He holds a BA from the University of California at Santa Barbara and an MBA from Indiana University.