====== Getting NCover and NUnit to play nicely with .NET 4.0 ======

{{ :blog:2011-06-23:coverage.png?300}}

Today I spent a great deal of time pulling my hair out in an attempt to get code coverage working for an application I'm developing at my place of employment.  In an effort to hopefully prevent myself and others from enduring the same degree of pain and suffering, I am documenting my findings below and how I got it working.

===== The background =====

Where I work, we use a continuous integration server known as CruiseControl.NET.  For those not familiar with what continuous integration is, it's essentially a paradigm in which every time a commit is submitted to version control, a server pulls that change and rebuilds the modified code, and then performs verious metrics on it (unit tests, code coverage, etc), and returns results.  This is extremely useful as it can immediately tell you if you have cause unit tests to fail or broken the build.

<note>
For those unaware, //unit tests// are essentially code snippets that are designed to test a specific piece of production code (typically one possible code path of one specific method/function), and //code coverage// is essentially a metric of how many lines of your production code are actually tested (i.e. run) by your unit tests.
</note>

The cool thing about CruiseControl.NET is that it not only runs all of these metrics for you, but displays nice little reports about them.  Well, that is it is //SUPPOSED// to.

===== The trail =====

NCover was not spitting out anything to the coverage report.  There would be a coverage report, but it would be empty.  This would cause all of the builds to report having 0% coverage, regardless of how many tests would run.

Debugging this was a nightmare, because of how everything is run.  Essentially the following happens when CruiseControl.NET fires off a build on this specific project:

  - CruiseControl.NET detects a source code change
  - CruiseControl.NET pulls down the latest source
  - CruiseControl.NET uses NAnt to fire off a build script
    - NAnt uses MSBuild to build the project
    - NAnt uses NCover to fire off the coverage report
      - NCover uses NUnit to fire off the unit tests
      - NUnit performs the unit tests
      - NCover analyses profiled information from when the unit tests were run
    - NAnt uses NCoverExplorer to fire off the human-readable coverage report
      - NCoverExplorer creates the human-readable report
  - CruiseControl.NET publishes all of the artifacts from the build

Now my problem was specifically with NCover, but NCover WAS working in some way, as the unit tests were being run, the problem was that none of the coverage was being reported.

===== The problem =====

After much digging, I descovered that NCover uses the .NET profiler to follow the code while it's executing in the unit tests and performs coverage in that way.  So I looked at the profiler.

The version of NCover we were using as 1.5.8, which was the last open community edition.  It was written some time ago; however, so it was designed for .NET 2.0 while our application was running on .NET 4.0.

//That was the problem.//

Apparently, .NET 4, by default, disables all profilers not explicitly built to run on .NET 4.  Since NCover (having been built ages ago) ran on .NET 2.0, it was also using a .NET 2.0 profiler.  This meant that the NCover profiler was being refused when attempting to connect to the .NET 4 application we were writing.

===== The solution =====

So now we get to the good part, how to get .NET 4 to shut the hell up and accept our .NET 2 profiler.

==== Activation policies and frameworks ====

The first step is to make sure that the application we are trying to perform coverage on uses the .NET 2 activation policy, while also running under the .NET 4 runtime.  This is done by editing the //*.exe.config// file.  In my case the application I was having NCover profile was NUnit, so I edited //nunit-console-x86.exe.config// and placed the following lines of code into its //<configuration>// block:

<code xml>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedRuntime version="v4.0.30319" />
  </startup>
</code>

==== Environment variables ====

Yes, you read that correctly.  In order to get this working, we have to have NCover running with some environment variables that tell .NET 4.0 to use a .NET 2.0 profiler compatibility setting.

Since I was running NCover using an NAnt build, I essentially added the following lines of code to my NAnt build:

<code xml>
<setenv name="ProfAPI_ProfilerCompatibilitySetting" value="EnableV2Profiler" />
<setenv name="COMPLUS_ProfAPI_ProfilerCompatibilitySetting" value="EnableV2Profiler" />
</code>

==== Specify framework and noshadow on the console ====

The last step is to disable shadowing and set your framework in the NAnt //<ncover>// task to .NET 4.0.  This is relatively easy:

<code xml>
<ncover program="${ncover.console}" 
		commandLineExe="${nunit.console}"
		commandLineArgs="/framework=4.0.30319 ${fixtures} /nologo /noshadow /xml:${output.dir}\reports\unit.tests\unit.tests.xml"
		workingDirectory="${output.dir}"
		coverageFile="${output.dir}\reports\coverage\${project::get-name()}.report.xml"
		verbose="true"
		excludeAttributes="CoverageExcludeAttribute"
		assemblyList="${assemblies}"
		logFile="${output.dir}\reports\coverage\${product.name}.ncover.log"
		logLevel="Verbose">
</code>

As you can see, on the command line I specify ///framework=4.0.30319// and ///noshadow//.  This forces the use of .NET 4 as well as disables shadow copying in NUnit, which can also break coverage.

===== If all went well... =====

At this point, if all went well then CruiseControl.NET should be able to build and produce coverage reports the way it is expected (assuming you have everything else set up properly, that is).

===== References =====

  * [[http://stackoverflow.com/questions/4921443/force-ncover-1-5-8-to-use-v4-framework-like-testdriven-net-does|Force NCover 1.5.8 to use v4 framework]] - the post where I finally found most of this information
  * [[http://msdn.microsoft.com/en-us/library/dd778910.aspx|Profiler Compatibility Settings]] - MSDN article about profiler environment variables
  * [[http://kiwidudeuk.blogspot.com/2007/04/ncover-problems-fixes-part-2.html|NCover Problems/Fixes]] - article where I discovered the /noshadow issue.

====== tl;dr ======

NCover and NUnit explode trying to perform coverage on a .NET 4 application.

You need to specify some env variables, ncover args, and nunit configs to get it working.

It was a real pain trying to find these answers.

~~NOTOC~~