Automate Codeplex Releases in your Powershell build and deployment scripts by Matt Wrock

codeplexI maintain a few projects on and one that I have been working on actively lately is Boxstarter. A sort of “Add-On” to Chocolatey that allows you to run complex (or simple) environment scripts completely unattended in a reboot/logon resilient manner.

Whenever I startup any project that I plan to develop beyond a hack or a spike, the very first thing I do is lay down test infrastructure along with build and deployment automation. These are NOT “nice to haves” and especially since I devote my “free” time to these projects, it is important that I make the most of that time and spend as little time as possible chasing regression bugs or mucking with the Release gymnastics of packaging things up and copying/uploading files, etc. It is too easy to get that stuff wrong and takes too long to do and test every detail by hand every time I want to push bits - not to mention that it is just not fun.

When I push a Boxstarter release, my download zip, click-once app  and home page are updated on my Azure web site by doing a GIT Push to Master. My Chocolatey packages are packed and pushed by a deployment script. So the one manual step has been uploading ,my download zip file to Codeplex and creating a Codeplex Release for it. Well it ends up there is a web service that codeplex maintains to facilitate the automation of this step. It is a SOAP service and therefore a bit of a pain to call directly, but Codeplex also has a C# SDK that makes working with this service very easy.

While the SDK is a .Net Assembly that was likely intended for C# based clients, I prefer to write my build and deployment scripts in Powershell. I believe that what happens in a deployment should be easily discoverable and that the script should serve as the “deployment document.” Compiled code does not lend itself well to that kind of transparency. I also don’t like to use XML for control flow and therefore avoid msbuild for all but the basic compile tasks. PSake is a great powershell tool I have been using for years to organize and execute build and deployment scripts. Every build task is encapsulated by a Powershell function called Task. Here is my Push-Codeplex task:

Task Push-Codeplex { Add-Type -Path ` "$basedir\BuildScripts\CodePlexClientAPI\CodePlex.WebServices.Client.dll" $releaseService=New-Object CodePlex.WebServices.Client.ReleaseService $releaseService.Credentials = Get-Credential `   -Message "Codeplex credentials"`   -username "mwrock" $releaseService.CreateARelease(`     "boxstarter",`     "Boxstarter $version",`     "Running the Setup.bat file will install Chocolatey if not present `      and then install the Boxstarter modules.", `     [DateTime]::Now,[CodePlex.WebServices.Client.ReleaseStatus]::Beta,`     $true,`     $true) $releaseFile = New-Object CodePlex.WebServices.Client.releaseFile $releaseFile.Name="Boxstarter $version" $releaseFile.MimeType="application/zip" $releaseFile.FileName="boxstarter.$" $releaseFile.FileType=`   [CodePlex.WebServices.Client.ReleaseFileType]::RuntimeBinary $releaseFile.FileData=[System.IO.File]::ReadAllBytes(`   "$basedir\BuildArtifacts\Boxstarter.$") $fileList=new-object `   "System.Collections.Generic.List``1[[CodePlex.WebServices.Client.ReleaseFile]]" $fileList.Add($releaseFile) $releaseService.UploadReleaseFiles("boxstarter", "Boxstarter $version", $fileList)}

This adds the types in the Codeplex.Webservices.Client assembly to the current Powershell session. I have downloaded  the SDK and put it in my source tree. Its too bad this is not packaged in its own Nuget package. Then I could use Package Restore to grab the bits from

Note that this assembly has a dependency on a couple Build assemblies. You could try to track these down separately but since I have Visual Studio installed, this gives me these dependencies.

The parent object handling all codeplex project commands takes a ICredentials to authenticate the client. I can simply use Get-Credential for this. This does require me to manually enter my password but that is fine for my script.If you had a more enterprise grade deployment that needed to fully script the authentication, that is very possible. After that I am free to create my release, and upload my file. This worked flawlessly for my project and now I can deploy my project with a single command.

Setup a new machine with just a URL and Chocolatey package by Matt Wrock

When I first started working on Boxstarter, a set of scripts that can stand up a complete, customized windows environment in a completely unattended manner, one of my key objectives was to be able to spin up a new environment with as few bits on hand as possible. Until now, Boxstarter achieved this by allowing you to create a portable Chocolatey package on a thumb drive or network share. I have never really been satisfied with that. It always seems when the time comes and I want to repave, that thumb drive or share is not around or in an up to date state. What I really want is just to type something brief and rememberable that would bring down all the bits I need to perform the setup (at least solving the “not around” part). This is now a reality.

Introducing The Boxstarter Click-Once Web Launcher

You can use any browser that supports Click Once applications. This is limited to IE unless you add extensions to Chrome or Firefox. You can also use either the Command Line or a Powershell console, both using the same command as long as IE is your default browser. The fact that this method is rather IE-centric is not much of a drawback since every Windows machine has IE as its default browser in the base install which is the key Boxstarter scenario.

launchAll you need to remember is the Boxstarter domain and the Chocolatey package that drives your machine setup. In the above example, the package is called…well…example. Boxstarter probes both the public gallery and the community boxstarter feed for a package ID named Example.

This first installs a Boxstarter Click-Once Bootstrapper.








Be aware that on Windows 8, the Smart Screen filter will supply a somewhat frightening message stating that you have downloaded software that is not yet trusted.






Even though the Boxstarter Web Launcher is signed with an Authoritative certificate, it has not “gained enough reputation” to be considered safe. In theory, this should eventually go away. In the meantime, you need to click the “More info” link to get this dialog:






Click the “Run Anyway” link to proceed. Next, unless you have UAC disabled, you need to give Boxstarter permissions to run with administrative privileges. Boxstarter needs these privileges in order to run all of the application installs, reboot and perform all actions you want it to to tweak your windows environment.







Next the normal Boxstarter session begins and prompts you for your windows account password:







Boxstarter stores the password you enter in a special place in windows designed specifically for this purpose. The password is stored in encrypted form and is never sent over the network or stored in plain text. This allows you to be automatically logged on whenever Boxstarter needs to reboot your system throughout the system installation.

If you want to install a package without the risk of reboot and without this prompt, you can invoke boxstarter using this URL:


Its all in the “/nr/”. This might be handy if you know no reboot is needed and perhaps you want to use Boxstarter because you know Chocolatey is not installed.

What happens next?

Boxstarter now installs Chocolatey and (if you are on windows 7 or server 2008 R2) .net4.0 if these are not already installed. It then proceeds to run the package you specified in the original URL. This is just a normal Chocolatey package. Boxstarter can run any Chocolatey package. What’s different when running the package in Boxstarter is the following:

  • Boxstarter shuts down the windows update service and if installed, the SCCM client. These can often interfere with software installations.
  • Boxstarter intercepts all Chocolatey install calls. If there is a pending reboot, boxstarter invokes a system reboot and ensures that it will be reinvoked first thing on startup.
  • Boxstarter examines all Chocolatey failures and if these are standard MSI Pending reboot failures, it will like above, restart your system and try again after restarting.
  • Boxstarter runs the entire installation as admin to avoid prompts for permissions elevation or errors complaining about insufficient permissions.
  • Boxstarter imports the Boxstarter.Winconfig powershell module making all of its CmdLets available that provide various helpers for customizing windows settings (enabling remote desktop, customizing windows explorer defaults, icon sizes, etc.). The most important feature in this module is the ability to run all critical windows updates. A routine requirement when setting up a new system.

What might a Chocolatey package look like for a Boxstarter install?

Here is an example:

Update-ExecutionPolicy Unrestricted Move-LibraryDirectory "Personal""$env:UserProfile\skydrive\documents" Set-ExplorerOptions -showHidenFilesFoldersDrives -showProtectedOSFiles -showFileExtensions Enable-RemoteDesktop cinstm VisualStudioExpress2012Web cinstm fiddler cinstm mssqlserver2012express cinstm git-credential-winstore cinstm console-devel cinstm skydrive cinstm poshgit cinst Microsoft-Hyper-V-All -source windowsFeatures cinst IIS-WebServerRole -source windowsfeatures cinst TelnetClient -source windowsFeatures Install-ChocolateyPinnedTaskBarItem "$env:programfiles\console\console.exe" copy-item (Join-Path (Get-PackageRoot($MyInvocation)) 'console.xml') -Force $env:appdata\console\console.xml Install-ChocolateyVsixPackage xunit Install-WindowsUpdate -AcceptEula

This is a relatively simple setup. Here is what will happen:

  • The user’s powershell execution policy will be permanently set to unrestricted.
  • The MyDocuments folder is moved to Skydrive (I always do this and it is incredibly convenient since my docs (including things like powershell profile) are synced between machines and makes the risk of a machine failure less ominous).
  • The Windows explorer settings are set to a developer sane level.
  • Remote Desktop is enabled.
  • Several typical developer applications are installed including Visual Studio and Sql server. When installed together these two often make for a “reboot perfect storm.” All of their dependent packages will be installed as well like GIT for poshgit and the .net 4.5 framework for Visual Studio.
  • Hyper-V, IIS and the Telnet client are installed
  • I “pin” Console, my favorite command line console, to the task bar and copy its configuration file with my preferred settings that I embedded in the chocolatey package to the directory where it expects to find it.
  • I install the XUnit test runner Visual Studio extension.
  • Lastly I install all critical windows updates.

Chances are this will include at least one or two reboots in windows 8 and several more in Windows 7. Boxstarter will reboot when needed and then simply rerun the package after restart. Chocolatey helps us from installing packages more than once. On subsequent runs it knows if we installed something previously and will just skip to the next install. This does mean that any custom script we add, and we can add any powershell we want, must be repeatable.

I thought Powershell 4’s Desired State Configuration solves this

Desired state configuration is the flagship feature shipping with Powershell 4 on Windows 8.1 and Server 2012 R2. It is no less than very cool and will indeed change the way we do machine configuration in Windows at last. It allows you to create declarative configuration files hat describe exactly how you want a machine to look and then tells one or more machines in your enterprise, look like this and do whatever it takes to make that happen.

The gaps that Boxstarter tries to fill here are:

  • Backward compatibility back to windows 7 and server 2008 R2.
  • Support for client systems. DSC is currently strongly targeting server infrastructure
  • Tooling. Eventually, I want people to be able to use Boxstarter who have no knowledge of powershell or any scripting for that matter.

What's next for Boxstarter?

I’ve been on a bit of a oss hiatus lately with my day job keeping me quite busy during non daylight hours, but I have had some time lately to do some polishing and also make some headway into another goal I have for Boxstarter: Easily deploying a Boxstarter install to a new VM. If you are familiar with Vagrant, a great VM environment setup platform for VirtualBox, VMWare and AWS then you can picture where I am going with this. I’d like to bring the same (or atleast very similar) capabilities to Hyper-V and windows Azure VMs.

This week I wrote a test script to help me automate testing Boxstarter packages and Boxstarter itself. Its just part of my PSake build file now:

task Test-VM -requiredVariables "VmName","package"{ $vm= Get-VM $VmName Restore-VMSnapshot $vm-Name $vm.ParentSnapshotName -Confirm:$false Start-VM $VmName $creds= Get-Credential -Message "$vmName credentials"-UserName "$env:UserDomain\$env:username" $me=$env:computername $remoteDir=$baseDir.replace(':','$') $encryptedPass= convertfrom-securestring -securestring $creds.password $modPath="\\$me\$remoteDir\Boxstarter.Chocolatey\Boxstarter.Chocolatey.psd1" $script= { Import-Module $args[0] Invoke-ChocolateyBoxstarter $args[1]-Password $args[2] } Write-Host "Waiting for $vmName to start..." do {Start-Sleep -milliseconds 100} until ((Get-VMIntegrationService $vm | ?{$ -eq"Heartbeat"}).PrimaryStatusDescription -eq"OK") Write-Host "Importing Module at $modPath" Invoke-Command -ComputerName $vmName-Credential $creds-Authentication Credssp -ScriptBlock $script-Argumentlist $modPath,$package,$creds.Password }

This takes a given VM and reverts it to its parent snapshot and then uses Powershell remoting to run the install. Take a look at this gist I wrote with a function to test pure Chocolatey packages. It includes some comments on how to setup the remoting.

I’d like to improve on this by cloning a VM from an existing VHD, then run the Boxstarter install on the VM taking over the mess of setting up the remoting and VM networking. We’ll see how that shapes up.


If you have the chance to try Boxstarter out, I’m very interested to hear how he experience is. What is confusing, what does not work or what you would really like to see added as a feature. Please feel free to utilize the Codeplex Discusion and Issues features to voice these. You can also find complete documentation on the documentation wiki.

Extract TFS Pending Changes to a zip file by Matt Wrock

Our TFS server was down today and I needed to get a Shelveset to a tester. Playing with the Power Tools PowerShell CmdLets I was able to basically pipe my pending changes to a zip file and give that to the tester.

Getting the Power Tools PowerShell Cmdlets

If you have Visual Studio Update 1 or Update two, you can download the TFS Power Tools from the Visual Studio Gallery here. If you have an older version of Visual Studio, download the Power Tools from the Microsoft Download Center here.

When you install the MSI, make sure to opt in to the PowerShell Cmdlets. The default options will not include the PowerShell Cmdlets!

Accessing the Cmdlets

After you have installed the power tools, you can launch the Power Tools Powershell console from the start menu item created during install. However if you are like me and have your own shell preferences, simply import them into your shell:

Import-Module "${env:ProgramFiles(x86)}\Microsoft Team Foundation Server 2012 Power Tools\  Microsoft.TeamFoundation.PowerTools.PowerShell.dll"

You may want to add this to your PowerShell Profile.

Extracting Pending Changes

Run the following commands from the root of your workspace to pipe all pending changes to a zip.

Add-Type -AssemblyName System.IO.Compression.FileSystem,System.IO.Compression$here= (Get-Item .).FullName$archive = [System.IO.Compression.ZipFile]::Open(  (Join-Path $here ""), [System.IO.Compression.ZipArchiveMode]::Create)get-tfsPendingChange | % {   [System.IO.Compression.ZipFileExtensions]::CreateEntryFromFile(    $archive, $_.LocalItem, $_.LocalItem.Substring($here.Length+1)  ) }$archive.Dispose()

This simply adds the .net types to the powershell session and uses them to create a ZipArchive and then calls the Get-TfsPendingChanges to get a list of all files with pending changes. This is sent to a Zip file which in this case is called and located in your current folder.

Requires .NET 4.5 and Powershell 3

The above zip file api makes use of the new, more friendly api for creating zip files. This will not work if you have .net 4.0 or lower. Also, since Powershell versions prior to 3.0 use the .NET 2 runtime, they will not be able to load .net 4.5 types. PowerShell 3.0 comes automatically unstalled on windows 8 and Server 2012. You may download and install the Windows Management Framework 3.0 here on Windows 7 or Server 2008 R2 to get Powershell 3.0 on those operating systems. You may get the .net 4.5 runtime here.

Easily Script Machine Reinstalls with Boxstarter by Matt Wrock

boxLogoAlmost a year ago now I started this small project, Boxstarter.  The project has sustained many interruptions as I have had busy spurts at work and involvements with other projects. It has gone through some serious refactorings moving from a scrappy script to a larger codebase organized into a few PowerShell modules and a suite of unit tests. A lot of its original code I moved over to Chocolatey and I plan to continue to contribute to Chocolatey where it makes sense. Chocolatey Rocks! So overall, I have probably rewritten this thing twice and now I feel it is far from finished but represents a nice base that makes “Box Starting” a much simpler and repeatable process.

Repaving Sucks

I think that pretty much says it. The idea of creating Boxstarter came to me when I had a SSD die and had to do a dreaded repaving of my machine shortly followed by installing several Windows 8 revisions from Consumer Preview to RTM. At the time, I was in the middle of a project where I was using powershell to automate a 50 page deployment document. So I knew if I can automate installing AppFabric, network shares, multiple web apps and other infrastructure settings, surely I can script the build of my own PC.

Then I found Chocolatey

So as I was looking in to how I could setup a fully functioning dev environment on one box to be just as  I left it on another, I inevitably discovered Chocolatey. Chocolatey is built on top of Nuget but instead of maintaining library packages for you dev project, it manages machine wide software package installations. This is good for several reasons:

  • Its plain simple to install apps that can be tedious to install on your own. Instead of hunting around the internet for the download page, forgetting to uncheck the animae tool bar download and waiting three minutes to click the next button, just type CINST <your app> and be done with it. Next time its time for a mega Visual Studio install session, save yourself and use CINST VisualStudioExpress2012Web.
  • Now lets say you have a bunch of apps you installed with Chocolatey and want to just update everything. Simply type CUP ALL.
  • The very best thing of all: create a “meta-package” or package.config and now you can install all of your apps in one go. Chocolatey just iterates the list and installs everything one by one along with all of their dependencies.

If you have not heard of or have not used Chocolatey, do yourself a favor and install it now.

What is Boxstarter? Chocolatey Tailored Specifically for Fresh Machine Installs

Chocolatey is awesome, but having done a TON of experimentation with automating new machine setups of all sorts of flavors, OSs and complexity, I have learned that setting up an environment can be much more than simply running a chain of installers.

Let me quickly list the benefits of Boxstarter and then I’ll dive into a few highlights:

  • Ensures the entire install session runs as administrator. This avoids occasional prompts to elevate your shell and limits it to just one at the beginning assuming you are not already running as admin.
  • Shuts down the Windows Update Service and Configuration Manager if installed during the install session. These can often interfere with installations causing installs to fail because either an update is blocking the install you are trying to run or they install patches that require a reboot before other software can be installed.
  • Can discover if there is a pending reboot and will reboot the machine and restart the install if asked to reboot. If written correctly, the install will pretty much start from where it left off. Further, Boxstarter can automatically log you in so you don’t have to stick around.
  • Boxstarter handles the initial installation of Chocolatey and if you are on a fresh win7 or server 2008R2, it will install .net 4.0 first which is a Chocolatey prerequisite.
  • Provides a bunch of helper functions for tweaking various windows settings.
  • Automates installation of critical windows updates.
  • Makes it easy to setup a local Boxstarter repo on your network so any connected machine can kickoff a install with one command.
  • Provides helper functions making it easy to create your own Boxstarter package.

The Boxstarter Killer Feature: Handling Reboots

I used to spend hours tweaking my install scripts, playing with ordering and various tricks to avoid having to reboot. There finally came a point when I realized this was pointless. Win8/Server2012 are a lot more reboot resistant but are still prone to them. Things get worse here when you are installing patches and more complicated apps like Visual Studio an/or SQL Server. I have realized that Reboots happen and can be unpredictable so the only thing to do is be able to deal with them.

The challenges are making sure the install scripts picks up right after restart, ensuring that the script does not spark a UAC prompt and block the setup, have it securely store your credentials so that it automatically logs back on after reboot but turns off auto logins after the script completes.

Boxstarter does all of these things. As a Boxstarter package author, you simply need to compose your packages to be repeatable. This means you should be able to run it again and again without error or data loss and ideally it should skip any setup processes that have already been run.

What is a Boxstarter Package?

Its just a Chocolatey package, but its intent is usually to either install a fresh environment or to lay down a complicated install chain that is highly prone to needing one or even several reboots. You can store them locally, on Chocolatey or on Myget or anywhere else you configure Boxstarter to look.

Show me the Code

First. Install Boxstarter.  The easiest way to do this is to install Boxstarter.Chocolatey from Chocolatey or download the zip from the CodePlex site and run the setup.bat. This installs all dependent modules and puts them in your user module path.

Next create a package, build it and deploy your repository to b consumed from anywhere in your network or even a thumb drive. Like this:

#After extracting on MYCOMPUTER
Import-Module $env:appdata\boxstarter\Boxstarter.Chocolatey\Boxstarter.Chocolatey.psd1
#Create minimal nuspec and chocolateyInstall
New-BoxstarterPackage MyPackage
#Edit Install script to customize your environment
Notepad Join-Path $Boxstarter.LocalRepo "tools\ChocolateyInstall.ps1"
#Pack nupkg
Invoke-BoxstarterBuild MyPackage

#share Repo
#Or Copy to thumb drive G
Copy-Item $Boxstarter.BaseDir G:\ -Recurse

#Logon to your bare Windows install
\\MYCOMPUTER\Boxstarter\Boxstarter Mypackage

#Enter password when prompted and come back later to find all your apps installed


Now lets look at what an install package might look like

Install-WindowsUpdate -AcceptEula
Update-ExecutionPolicy Unrestricted
Move-LibraryDirectory "Personal" "$env:UserProfile\skydrive\documents"
Set-ExplorerOptions -showHidenFilesFoldersDrives -showProtectedOSFiles -showFileExtensions

cinstm VisualStudioExpress2012Web
cinstm fiddler
cinstm mssqlserver2012express
cinstm git-credential-winstore
cinstm console-devel
cinstm skydrive
cinstm poshgit
cinstm windbg

cinst Microsoft-Hyper-V-All -source windowsFeatures
cinst IIS-WebServerRole -source windowsfeatures
cinst IIS-HttpCompressionDynamic -source windowsfeatures
cinst IIS-ManagementScriptingTools -source windowsfeatures
cinst IIS-WindowsAuthentication -source windowsfeatures
cinst TelnetClient -source windowsFeatures

Install-ChocolateyPinnedTaskBarItem "$env:windir\system32\mstsc.exe"
Install-ChocolateyPinnedTaskBarItem "$env:programfiles\console\console.exe"

copy-item (Join-Path (Get-PackageRoot($MyInvocation)) 'console.xml') -Force $env:appdata\console\console.xml

Install-ChocolateyVsixPackage xunit
Install-ChocolateyVsixPackage autowrocktestable


Whats going on here?

Boxstarter installs critical updates, sets your powershell execution policy to unrestricted, makes windows explorer usable, installs some great apps, installs some of your favorite windows features, moves your Documents library to skydrive (I love this for the truly portable desktop), installs your favorite VS extensions and sets up things like pinned items and task bar size preference.

A lot of this functionality comes to you compliments of Chocolatey and others are specific to Boxstarter.

What’s Next?

As I see it, this is just the absolute base functionality so far. There is so much more to be added to make an installation process truly seamless. Here are some features I plan to begin soon:

  • Create a Boxstarter package automatically based on software already installed and windows features turned on to mimic these on another machine.
  • While Boxstarter can be installed and run either on bare metal or a VM, I want to make Boxstarter the powershell/hyper-v equivalent of Vagrant. Making the deployment of scripted VMs simple.
  • Add a one-click installer making a an easy one liner install command possible remotely.

There is a lot of Boxstarter functionality I have not covered here. I plan to blog fairly regularly providing brief posts describing various ways Boxstarter can augment your automation. If you want to learn more now. Checkout the Boxstarter Codeplex site which has complete documentation of all features and commands.

Is changing an API or design solely for testability a good practice? by Matt Wrock

imageThere is a personal story behind this topic that I want to share. About five years ago I heard about this thing called Test Driven Development (TDD). For anyone unaware of this, it is where you write failing tests first and then write “the code” later to make them pass. It immediately struck me as interesting and the more I learned about it the more I liked it. It seemed like much more than a means of testing code but a design style that could transform the way we write software.

As I actually began to practice this discipline, it did completely change the way I write code and approach design. It has been very challenging and rewarding. I don’t at all consider myself polished or advanced but I have drunk the Kool-Aid and hey, who doesn’t like Kool-Aid!? I cover more specifics later but please, let me reminisce.

At the time when I was discovering this technique, I purposely sought out a team of developers who were TDD practitioners. As someone who is self taught and prefers to learn on my own, this was an area I knew I needed to learn from others to better get my head around the patterns. So for almost three years I was immersed with my new team. We often discussed the virtues of strong unit test coverage and TDD. When you roll with a group that all share a common set of values, it is easy to feel “right” and take some nuances for granted. Even if they are right.

Maybe the world IS flat

Six months ago I changed teams. The new team was different from my former team. One difference was a lack of unit testing. There were lots of tests but they were essentially integration tests and took a long time to run. So long that they could not be run as part of the build. Tests were always written after coding and one reason why was that the code itself was nearly impossible to test using typical Unit Testing techniques.

One of my initial thoughts going in was that this would be an excellent opportunity to make a difference for the better. I still believe this but I was very surprised at how some of my techniques and the patterns that I had come to embrace were questioned and given some rather skeptical critiques. Suddenly it felt that the things I had come to value in standards of design and build methodologies was a currency not honored in this foreign land.

Now it would have been one thing if I was working for some mediocre outfit of 9 to 5er developers. However many of these were people I consider to be very smart and passionate about writing good software. It was like having a bunch of Harvard grads insisting the earth is flat and watching yourself begin to question if the world really is round. Sure does look flat when you really look at it.

Not everyone likes chocolate and some who don’t are smart

Sidebar: I just learned today that Jonathan Wanagel, who runs Codeplex and is one of the smartest people I know in the whole wide world does not care much for chocolate. Interesting…

What is this Hippie code?

Here is a list of many of the things that turned some people off:

  • Interfaces – adding too many types to the API, kills Visual Studio’s F12 and overkill when there is only a single production implementation.
  • Unsealing classes, adding virtual methods or making some private/internal types public – Now we have to support a larger surface area and make sure customers do not get confused or shoot themselves in the foot.
  • Eliminating static classes – now I need to instantiate a class to access a simple utility or helper method.
  • Test methods with long method names resembling a sentence describing what is being tested –that’s just weird.
  • A general concern expressed by many - We should not have to change code and especially an API just to add testability.

Some of these concerns are very valid. Honestly they are all valid. Even though I completely disagree with some of the opinions I have encountered, it must be remembered that introducing new ideas will be naturally distasteful to some if they do not understand the intent and it is therefore incumbent upon the bringer of the new ideas to clarify and articulate why such ideas might have value. I’m not claiming “mission accomplished” but this forced me to do a lot of “back to basics” research to understand how to communicate the value. While I feel the value deep in my bones, it has been very challenging to learn how to express the value to others.

Lets play Devil’s Advocate

Before I dive into the responses I have developed and am still developing for the above reactions to a different style of design I would like to spend some time defending the critics and skeptics. I work with very intelligent people and I respect the fact that they demand to understand why someone suggests a change in style and a radical change in some respects.

Personally I spent more than half my career not practicing TDD or writing proper unit tests. I too was very conservative about what my API exposed. I liked to write APIs with few classes. I liked “noun” style classes (employee, order, etc) with their own persistence logic and static Create methods. I thought this was elegant and a lot of others think so too. And honestly, I wrote some code that made some people a lot of money during this time.

Sealed classes, internal methods, oh boy!

Among the crowd I run with on the twitters, there is a lot of nay saying around sealed classes and internal methods. Don’t expect me to be changing my Add New Class template to create them sealed by default but I do think this deserves some thought. My background is largely in web technologies. This often means that my server code will never be exposed to anyone outside of my team. It is easy to adopt much looser rules around api exposure when you are the only consumer.

I am also involved in open source software. We are a small cadre of developers and we are not representative of the average developer. I know. That sounds elitist but it is true. I’m used to reading source code in lieu of documentation (not necessarily a good thing). I’m perfectly comfortable pulling down someone else’s code using any of a half dozen source control providers. And like many of my OSS peeps, I get extremely annoyed when I am trying to work with an API, find some code that looks to be just what I need and then see it is not accessible. I’d rather have a larger api that provides me heightened flexibility and extensibility over a smaller and easier to understand API – within reason of course.

However, when you work on software that physically ships to enterprise customers, you really do need to broaden your view. I might work for Microsoft, but if you call me an enterprise customer, I’ll cry. The fact is, it is important to understand your audience. Most developers don’t want to concern themselves with the innards of your code. That’s why they buy it. It should do everything they need it to do and be easy to figure out and difficult to misuse. No matter who your audience is, the public API is one of the most important things to get right. It should read like documentation and be self explanatory. Sometimes this means putting a curtain over large parts of your codebase. I’m still coming to terms with this, but I do believe it is a reality that deserves attention.

Testability for its own sake – That’s just fine

I used to feel an uneasiness when discussions would take this turn into warnings about the dangers of making code testable simply for its own sake. I would feel a sense of guilt about asking others to work harder just to make things easier to test. I’m over that. This is like arguing against quality for its own sake or simplicity just to be simple. If I have to tweak an API to make it testable, we have to remember that we are not only doing ourselves a favor but everyone who will be using our software can now test around it as well.

While TDD and similar practices have been fairly mainstream in other communities for a while, it is becoming more so in the Microsoft dominated technologies (where I work) and much more so than it was just a few years ago. We need to understand that testability ships as a feature to our customers. Software that is difficult to test is not just perceived as a nuisance but its overall quality can be called into question by virtue of a lack of testability.

So what is so great about testability?

Perhaps I’ve gone far too long in this post before championing the virtues of testability not to mention some clear examples of what it is. I can hear others questioning: what are you talking about? We have QA staff and huge suites of test automation. Testability? Lets not get carried away.

Yes. Testability is very much about testing the code that you have written. Specifically, I am referring to the ability to test small single units of code (an IF block for example) without the side effects of surrounding code. I might have a method that queries a database for a value, sets the state of the application depending on the value queried and logs the result. I want to be able to write tests that can check that I set the application state appropriately but not that I got the right data from the database or that I successfully logged the activity. I’ll write other tests for that.

Having a code base well covered with these kinds of tests can create (but does not guarantee) a very high quality bar and allow developers to spend more time writing features and less time finding and fixing bugs. The more logical paths your code can take, the more important this is. It is easy to innocently introduce a small change and inadvertently break several logic paths. This level of code coverage is your safety net from bug whack-a-mole. Each release without this coverage brings increased surface area for bug creation until the cyclomatic complexity drives you to a saturation point and now you spend most of your time addressing bugs.

Good test coverage and code that is easy to test invites exploration and experimentation. It is the fence that keeps us from touching the third rail. Low coverage incites fear into the hearts and minds of good developers. “Hmm. That sounds like a really interesting approach but we don’t dare touch that code because it is core to our business and we cant afford for it to break.“

Testable code is more about good design than a test automation arsenal

This is something that can sound odd to the unindoctrinated. At least it was not what originally attracted me to TDD but it IS what has kept me here.

In order to write code that is easy, let alone possible, to test with this kind of granularity, one must enforce a strict separation of concerns because you want to test no more than a single concern at a  time. This may produce code that looks different to many teams and might look awkward at first glance. The code is more likely to have these traits:

  • More types. More classes, more methods and more interfaces. As one ensures that each type contains a highly focused and intent driven purpose. Rather than having a person class that not only represents the person but also does a bunch of stuff to and with a person, one may now have several classes that look more like verbs than nouns to represent different interactions with a person.
  • Smaller types and smaller methods. This goes hand in hand with the above. It does not take a genius (that’s why I figured it out) to discover that it is tough to test a method that does 20 things. And guess what? It is easier to read and understand too.
  • More layers of abstraction and points of extensibility. This may coincide with the mention of more interfaces. As you tease out corners of code to test, you need to be able to apply protective tape over surrounding machinery that should remain untouched by the test. This may be because these surrounding areas talk to out of process systems that would bog down the performance of a test or engage in complex logic that manipulates values that must interact with the code under test. It is easier to “plug in” lighter weight machinery that acts on data very predictably, repeatably and quickly. The use of interfaces, dependency injection and mocks/stubs/fakes/etc come into play here and may make one not used to them feel out of water or like they are over engineering. One may react that this abstraction seems silly. Why create an INamingService when we only have one naming service. First, that is a fair point and should not be ignored. It is possible to over engineer and you need to decide what level of abstraction your scenario calls for. That said, once you gain competence coding in this manner, you often find ways to exploit these abstractions into rich composition models that would not have been possible given a more monolithic class structure.
  • A larger surface area to the API. This is what many find the hardest to come to terms with and they should. There is A LOT to be said for a simple API and testability does not necessarily make this fate an inevitability. However it does make it more likely. With an application having more “building blocks” there may be a greater number of these blocks to interact with one another and exposing those interactions to the consumer may very well be a good thing. Also, just like you, your consumers writing code around your API may demand testability and the ability to abstract away all exposed blocks.

This design style facilitates change, improvement and happy developers

Code that is easy to compose in different ways is easier to change. This kind of a model allows you to touch smaller and more isolated pieces of your infrastructure, making change a less risky and dreaded endeavor. A team that is empowered to change and improve their code more rapidly makes for a happier team, happier business stake holders and happier customers.

Use modern tools for modern programming techniques

Ok. I’m gonna call TDD (because that’s usually what this methodology becomes more or less) a “modern programming technique.” There are tools out there that are designed to make these practices easier to implement. As a primarily C# developer, these tools include Resharper, an IOC for dependency injection, a good mocking framework and a modern test runner like XUnit.

Resharper makes a lot of the refactorings like extracting interfaces and finding interface implementations and their usages easier to discover. It provides navigational aides making work in a code base with more types much easier.

An IOC facilitates the creation of these types and makes it easier to manage either swapping out one type for another or discovering all implementations of a given type. One may find constructors with several types being injected which would be extremely awkward to “new up.” With a properly wired IOC, it handles the newing for you. Almost all IOCs (don’t create your own) provide solid lifetime management as well which provide the utility of singleton classes but without their untestability.

The wrong question

So lets return to the core question of this post. Is changing an API or design solely for testability a good practice? I would argue that we have not properly phrased the question. Begin by exploring your definition of “testability” and you may well discover that the “sake of testability” is not nearly all that you are after.