Cook Computing

Faking In Visual Studio 11

March 10, 2012 Written by Charles Cook

The Problem

Dealing with Now and why I'm almost done with C# and Java — Karl Seguin's post about how the way you code is a by-product of the progamming language you use — discusses how difficult it is to test unit code like this in C#:

var audit = new Audit
{   
    UserName = user.Name,    
    Dated = DateTime.Now,    
    //...
};

The following test might work or not, depending on how quickly the code runs:

public void ItSetsTheAuditTimeToRightNow()
{    
    var audit = CreateAuditItem(new User{Name = "Leto"});  
    audit.Dated.ShouldEqual(DateTime.Now);}   
}

Unfortunately it's difficult to mock DateTime.Now because it's a static method, which leads to more complicated solutions such as injecting an abstract dependency into objects or using a C# lambda such as Ayende's SystemTime class:

public static class SystemTime
{
    public static Func<DateTime> Now = () => DateTime.Now;
}

This allows you to write test code like this:

SystemTime.Now = () => new DateTime(2000,1,1);
repository.ResetFailures(failedMsgs); 
SystemTime.Now = () => new DateTime(2000,1,2);
var msgs = repository.GetAllReadyMessages(); 
Assert.AreEqual(2, msgs.Length);

Another solution mentioned in the post's comments is to use NUnit's Within:

Assert.That(audit.Dated, Is.EqualTo(DateTime.Now).Within(new TimeSpan(0, 0, 0, 0, 50)));

This works, sort of, but isn't a general solution to this type of problem.

Visual Studio 11 Fakes

Some of the comments mention TypeMock Isolator and the Moles project from Microsoft, and it so happens the Visual Studio 11 beta reveals that Moles has been productized into Visual Studio as the Fakes Framework. This can inject two types of dummy implementation into unit tests: stub types for interfaces and overridable methods, and shim types for static and non-overridables methods:

Stub types Stub types make it easy to test code that consumes interfaces or non-sealed classes with overridable methods. A stub of the type T provides a default implementation of each virtual member of T, that is, any non-sealed virtual or abstract method, property, or event. The default behavior can be dynamically customized for each member by attaching a delegate to a corresponding property of the stub. A stub is realized by a distinct type which is generated by the Fakes Framework. As a result, all stubs are strongly typed.

Although stub types can be generated for interfaces and non-sealed classes with overridable methods, they cannot be used for static or non-overridable methods. To address these cases, the Fakes Framework also generates shim types.

Shim types Shim types allow detouring of hard-coded dependencies on static or non-overridable methods. A shim of type T can provide an alternative implementation for each non-abstract member of T. The Fakes Framework will redirect method calls to members of T to the alternative shim implementation. The shim types rely on runtime code rewriting that is provided by a custom profiler.

Delegates Both stub types and shim types allow you to use delegates to dynamically customize the behavior of individual stub members.

Faking DateTime

To test the DateTime code, create a unit test project and right click on one of the referenced assemblies in Solution Explorer. This displays a context menu which has an "Add Fakes Assembly". Select this and two more referenced assemblies are automatically added to the project:

  • Microsoft.QualityTools.Testing.Fakes
  • Microsoft.VisualStudio.QualityTools.UnitTestFramework.10.0.0.0.Fakes.

Visual Studio will automatically generate a file called Microsoft.VisualStudio.QualityTools.UnitTestFramework.fakes in a directory in the project called Fakes. This XML file is used to configure the assembly for which fakes are generated and the namespaces and types that are included. We want to generate a shim type for DateTime so we can change the file to specify the mscorlib assembly:

<Fakes xmlns="http://schemas.microsoft.com/fakes/2011/">
  <Assembly Name="mscorlib" />
</Fakes>

Building the project results in Visual Studio creating an assembly containing the fake types in the FakesAssemblies directory. We need to then add a reference to this assembly so we can use the fake types in our test.

So, say we have this code under test:

public class Audit
{
    public string UserName { get; set; }
    public DateTime Dated { get; set; }
}

public static class TestClass
{
    public static Audit CreateAuditItem(string userName)
    {
        var audit = new Audit { UserName = userName, Dated = DateTime.Now };
        return audit;
    }
}

We can now write this unit test:

using System;
using System.Fakes;
using Microsoft.QualityTools.Testing.Fakes;
using Microsoft.VisualStudio.TestTools.UnitTesting;

[TestClass]
public class AuditTests
{
    [TestMethod]
    public void ItSetsTheAuditTimeToRightNow()
    {
        // wrap the test code in a ShimsContext to control 
        // the lifetime of your shims
        using (ShimsContext.Create())
        {
            // hook delegate to the shim method to redirect 
            // DateTime.Now to return January 1st of 2000
            ShimDateTime.NowGet = () => new DateTime(2000, 1, 1);

            var audit = TestClass.CreateAuditItem("Leto");
            Assert.AreEqual(audit.Dated, (DateTime.Now));
        }
    }
}

Calling ShimsContext.Create() within the context of a using statement means that the shim will be de-registered before the test function exits. If this is not done the shim will remain active for subsequent test and so might cause them to run in an unexpected way.

Although this doesn't address the limitations of C# that Karl described in his post (assuming you see them as limitations), at least we're not having to modify the original code using DateTime.Now to make it testable. Once the fakes are configured in a test project writing tests using the fakes is straightforward.

Testing Time

While Karl's post was not really about the problem of mocking DateTime.Now in itself, it is a good example of a non-deterministic test that can cause problems. Martin Fowler blogged about this type of test in his post Eradicating Non-Determinism in Tests, in particular the issues associated with testing time related functionality:

Few things are more non-deterministic than a call to the system clock. Each time you call it, you get a new result, and any tests that depend on it can thus change. Ask for all the todos due in the next hour, and you regularly get a different answer.

The most important thing here is to ensure that you always wrap the system clock with routines that can be replaced with a seeded value for testing. A clock stub can be set to particular time and frozen at that time, allowing your tests to have complete control over its movements. That way you can synchronize your test data to the values in the seeded clock.

Always wrap the system clock, so it can be easily substituted for testing.

One thing to watch with this, is that eventually your test data might start having problems because it's too old, and you get conflicts with other time based factors in your application. In this case you can move the data, and your clock seeds to new values. When you do this, ensure that this is the only thing you do. That way you can be sure that any tests that fail are due to time-movement in the test data.

Another area where time can be a problem is when you rely on other behaviors from the clock. I once saw a system that generated random keys based on clock values. This systems started failing when it was moved to a faster machine that could allocate multiple ids within a single clock tick.

I've heard so many problems due to direct calls to the system clock that I'd argue for finding a way to use code analysis to detect any direct calls to the system clock and failing the build right there. Even a simple regex check might save you a frustrating debugging session after a call at an ungodly hour.