CI Tools and Best Practices in the Cloud

Continuous Integration

Subscribe to Continuous Integration: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Continuous Integration: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn

Continuous Integration Authors: Flint Brenton, Elizabeth White, Liz McMillan, Pat Romanski, Stackify Blog

Related Topics: Continuous Integration

Blog Feed Post

Are Your Continuous Tests Too Fragile for Agile?

fragile_agileWith a fragile test suite, the Continuous Testing that's vital to Agile development just isn't feasible. If you truly want to automate the execution of a broad test suite—embracing unit, component, integration, functional, performance, and security testing—during continuous integration, you need to ensure that your test suite is up to the task. How do you achieve this?

Some best practices are to ensure that tests are: 

  • Logically-componentized: Tests need to be logically-componentized so you can assess the impact at change time. When tests fail and they're logically correlated to components, it is much easier to establish priority and associate tasks to the correct resource.

  • Incremental: Tests can be built upon each other, without impacting the integrity of the original or new test case.

  • Repeatable: Tests can be executed over and over again with each incremental build, integration, or release process.

  • Deterministic and meaningful: Tests must be clean and deterministic. Pass and fail have unambiguous meanings. Each test should do exactly what you want it to do—no more and no less. Tests should fail only when an actual problem you care about has been detected. Moreover, the failure should be obvious and clearly communicate what went wrong.

  • Maintainable within a process: A test that's out of sync with the code will either generate incorrect failures (false positives) or overlook real problems (false negatives). An automated process for evolving test artifacts is just as important as the construction of new tests.

  • Embedded within a prescriptive workflow based on results: When a test does fail, it should trigger a process-driven workflow that lets team members know what's expected and how to proceed. This typically includes a prioritized task list.

For more tips, see the  Code Curmudeon's slide deck for his  Are Your Continuous Tests Too Fragile for Agile? session from the recent Better Software Conference.

Read the original blog entry...

More Stories By Wayne Ariola

Wayne Ariola is Vice President of Strategy and Corporate Development at Parasoft, a leading provider of integrated software development management, quality lifecycle management, and dev/test environment management solutions. He leverages customer input and fosters partnerships with industry leaders to ensure that Parasoft solutions continuously evolve to support the ever-changing complexities of real-world business processes and systems. Ariola has more than 15 years of strategic consulting experience within the technology and software development industries. He holds a BA from the University of California at Santa Barbara and an MBA from Indiana University.