CI Tools and Best Practices in the Cloud

Continuous Integration

Subscribe to Continuous Integration: eMailAlertsEmail Alerts newslettersWeekly Newsletters
Get Continuous Integration: homepageHomepage mobileMobile rssRSS facebookFacebook twitterTwitter linkedinLinkedIn


Continuous Integration Authors: Elizabeth White, Yeshim Deniz, Liz McMillan, Pat Romanski, Mehdi Daoudi

Related Topics: Continuous Integration

Continuous Integration: Article

.NET Interview — Heard on Hanselminutes

Interview with Web developer and technologist Scott Hanselman

CF: And that's what you need at the end of the day. Have you seen Adam Cogan's Code Auditor that we talked about on .NET Rocks last week?

SH: Absolutely! There are lots of great tools like Adam's auditor. FxCop is similar. Adam's app works on code. FxCop works on Reflection. These are the kinds of things that you might want to build in. It would be a really great thing if Adam created XML as an output of his Code Auditor, I don't know if he does, that could then be read in and included in a report and CCNet would automatically be able to handle that.

CF: And, by the way, code auditor is at shrinkster.com/bmd.

SH: Cool! So if I can add NUnit doing automated testing, and I can do FxCop, what are some other things that I can do? Well, NUnit of course is a testing framework and there are also things like MBUnit.

CF: Now the automated builds aren't going to test automatically unless you have test cases, isn't that right, so you have to use some sort of test-generation tool to begin with?

SH: Well, not necessarily a test-generation tool but some kind of testing methodology. It would be silly to have things build and not know if they are healthy.

CF: Sure. SH: So one of the things that happens is: when you have to build a dozen times a day you start to crave this feedback. Is this a healthy product that we've made? We know it keeps building, we can look and see, oh, look, for example, the Voyager SDK project that I worked on has a 75% success rating on builds. That means it builds successfully three out of four times except if a test fails. Let's say it compiles but the test fails. We count that as a failed build. That's a fundamental part of Continuous Integration because building is easy. Building a testable integrated application and popping an installer out at the end is more difficult.

CF: Right.

SH: We can also set it up so that certain FxCop rules that we feel strongly about will break the build.

CF: How often, Scott, has the process turned out a "good build" and then it turns out you find bugs anyway?

SH: Probably half the time. I mean you still need good QA people.

CF: That's the point I wanted to make. That all this testing framework and stuff doesn't take the place of somebody actually sitting down and testing it manually.

SH: Exactly, and that QA person more and more at Corillian is pushing that test data back into the product.

CF: Cool! SH: Ideally, we would actually build the entire thing, do a functional NUnit test, then we would actually do a silent install of the product as part of the build, and then do an end-to-end integration test. And while NUnit does easy functional testing, there are tools like NUnitASP and NunitForms, you can get those at nunitforms.sourceforge.net and nunitasp.sourceforge.net. These are extensions to NUnit that let you test WinForms - actually like push buttons and poke around on a WinForm - or test ASP.NET Forms.

At Corillian we have been using a tool called WATIR, Web Application Testing in Ruby, and I have a little article on it up on my blog at shrinkster.com/bm8. If you go up on my blog and search for Watir, you'll find not only stuff that I've done with Watir but stuff that one of our guys at Corillian by the name of Travis Illig has done to extend Watir. We have another guy named Dustin Woodhouse who's extending Watir and NUnit such that this Ruby-based scripting language will actually create NUnit tests and NUnit results.

CF: Now a lot of people might not know what Ruby is.

SH: So, Ruby is a scripting language that a lot of people find very beautiful in its style. Feels kind of like Python; it feels kind of like Scheme, it feels kind of like C++. It's just one of those languages that feels nice in your mouth. There's actually a fellow who did a presentation where he had no bullet points. He only had Ruby code because he felt that the beauty of the code spoke enough. It's that kind of code. A gentleman named John Lam at iunknown.com believes very strongly in the beauty of Ruby. And every once in a while he'll have epiphany while he is writing Ruby and then he'll post it on his blog: "Oh! I took these six lines of code and turned into that two lines of code in Ruby and I'm just enamored with Ruby today."

Then Watir is an IE, an Internet Explorer wrapper for Ruby so that you can write these kinds of tests. And then I wrote a little cheapy application called Watir Maker that would basically record your experience while you're inside of IE and it would basically write out the beginnings of a Watir test. People are looking to extend that.

CF: So you mentioned building installers as part of CI. What are some tools to automate that process?

SH: So at the simplest level, of course, Visual Studio supports building installers from the development environment itself. For a lot of our stuff we just actually shell out and launch the deployment project that comes with Visual Studio from the command line. But for the times that that doesn't work you can use a number of different toolsets. You can use command-line versions of InstallShield, for example.

But since the Continuous Integration world and the CruiseControl world are so focused on getting a lot of great work done with Open Source tools, the tools you will often find are things like Windows Installer XML or WIX, that is at sourceforge.net/project/wix or the Nullsoft Scriptable Install System, NSIS, that used to be called the PiMP. This is the tool that the guys that made Winamp used to install Winamp. This is a very elegant install system at nsis.sourceforge.net. So you can call these different installer makers at the end of your NantScript and if you wanted to take it to the next level then you would do a remote deploy, or remote install of your application, and then start doing integration testing.

Some of the other things that you can stick at the end of your build once you have the ability to do all these things that framework are doing things like development metrics like what's called Cyclomtic Complexity. Cyclomatic Complexity is a measure of how many different paths through a particular function there are. Let's say that if there's a function called Foo that takes a Boolean. It has one input and it can be false or true. There are two ways through that particular method so then its complexity might be considered two.

There are some papers that are written about Cyclomatic Complexity and there are great tools like DevMetrics from a company called Anticipating Minds at shrinkster.com/bm9. They have a command-line tool that will go through your entire source code base and tell you the complexity of your particular project on a method-by-method basis. And then you can flag things in the build and, say, wow, anything that has a complexity of 10, indicating that there are 10 or more ways through this method, which is arguably a number that is greater than the number of paths that a human can comfortably store in his head, right?

CF: Right.

SH: I can barely hold my phone nuber and my wife's phone number. I can't hold 10 different paths through some code. That would be the time then to kick it out - we need to have a code review. Now whether you choose to break the build or not it's really up to you. But these are some [of the] kinds of feedback you can get that you couldn't before.

Another one is the tool called Simian, which is a similarity-based tool. It's actually kind of funny. It looks for copy[ing and] pasting in your source code base. It's at shrinkster.com/bma. Simian is actually written in Java but it will work on .NET code and it will find places where, say, you and I are working on the same project and I found a For loop that I really liked. I copied and pasted it but edited a little bit. And it will go and say, you know this is suspiciously similar, I think this is a candidate for refactoring. We did this on dasBlog and I think we found something like 14% or 15% of it was just complete duplication.

CF: Mark Miller told me once that Developer Express's refactor looks at Cyclomatic Complexity and that's one of the main characteristics they use to determine whether refactoring is necessary.

SH: Totally. Mark Miller being an expert on refactoring generally with all the refactors that he put into Refactor and into CodeRush. Having the nice chart and graph that you get as a toolbox in the CodeRush product is pretty powerful stuff. That's a very visceral way of finding out which function in your application kind of sucks the most.

Listen to the entire interview online at www.hanselminutes.com. The Hanselminutes Podcast is produced weekly by Pwop Productions, a podcasting services company at www.pwop.com.

More Stories By .NETDJ News Desk

.NETDJ News Desk monitors Microsoft .NET and its related technologies, including Silverlight, to present IT professionals with news, updates on technology advances, business trends, new products and standards, and insight.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.