Cook Computing

Unit Testing With ExpectedException

January 8, 2012 Written by Charles Cook

@mentalguy must have been having a frustrating day:

Mental Guy tweet

I've come across this problem with .NET code. Often a single exception type will cover several different error conditions and so when writing the corresponding unit tests it's tempting to assert on the exception's Message property. Of course this is bad because it assumes the text of the message won't be changed.

NUnit

NUnit encourages the checking of exception messages when using the ExpectedException attribute, for example

[ExpectedException(typeof(ArgumentException), ExpectedMessage="expected message" )]

but suggests that this is not good practice by allowing you to match on a substring or a regular expression:

public enum MessageMatch
{
  /// Expect an exact match
  Exact,    
  /// Expect a message containing the parameter string
  Contains,
  /// Match the regular expression provided as a parameter
  Regex
}

For example, this is for a test that passes only if an ArgumentException with a message containing "unspecified" is received.

[ExpectedException(typeof(ArgumentException), 
  ExpectedMessage="unspecified", MatchType=MessageMatch.Contains)]
public void TestMethod()
{
    ...
}

It's better to have some way of specifying an error code which can be checked independently of the message, for example say we have an exception base class supporting an error code from which we derive custom exception classes:

public class ExceptionBase : Exception
{
  public ExceptionBase(int errorCode, string message)
    : this(errorCode, message, null)
  {
  }

  public ExceptionBase(int errorCode, string message, Exception innerException)
    : base(message, innerException)
  {
    ErrorCode = errorCode;
  }

  public int ErrorCode { get; private set; }
}

public class MessageException : ExceptionBase
{
  public MessageException(int errorCode, string message)
  : base(errorCode, message)
  {
  }

  public MessageException(int errorCode, string message, Exception innerException)
  : base(errorCode, message, innerException)
  {
  }
}

We can test for exceptions with a particular error code like this:

[TestFixture]
public class Tests
{
  [Test]
  public void Test1()
  {
    MessageException ex = Assert.Throws<MessageException>(() => 
      {
        // code which is expected to throw the exception
        // ...
      }
    );
    Assert.AreEqual(ErrorCodes.HEADER_MISSING, ex.ErrorCode); 
  }
}

Visual Studio

Visual Studio does the right thing and doesn't have a built-in way of checking the exception message. You can even use your own attributes derived from its ExpectedExceptionBaseAttribute class. This allows us to implement an attribute which can be used to check the error code in exception classes derived from the above ExceptionBase class:

[AttributeUsage(AttributeTargets.Method, AllowMultiple = false, Inherited = true)]
public sealed class ExpectedExceptionWithErrorCode : ExpectedExceptionBaseAttribute
{
  public Type ExpectedException { get; set; }
  public int ExpectedErrorCode { get; set; }

  public ExpectedExceptionWithErrorCode(Type expectedException)
    : this(expectedException, 0, "")
  {
  }

  public ExpectedExceptionWithErrorCode(Type expectedException,
    int errorCode)
    : this(expectedException, errorCode, "")
  {
  }

  public ExpectedExceptionWithErrorCode(Type expectedException,
    int errorCode, string noExceptionMessage)
    : base(noExceptionMessage)
  {
    if (expectedException == null)
      throw new ArgumentNullException("exceptionType");
    if (!typeof(Exception).IsAssignableFrom(expectedException))
      throw new ArgumentException("Expected exception type must be "
          + "System.Exception or derived from System.Exception.", 
        "expectedException");
    ExpectedException = expectedException;
    ExpectedErrorCode = errorCode;
  }

  protected override void Verify(Exception exception)
  {
    if (exception.GetType() != ExpectedException)
    {
      base.RethrowIfAssertException(exception);
      string msg = string.Format("Test method {0}.{1} "
          + "threw exception {2} but {3} was expected.",
        base.TestContext.FullyQualifiedTestClassName, base.TestContext.TestName,
        exception.GetType().FullName, ExpectedException.FullName);
      throw new Exception(msg);
    }
    ExceptionBase ex = exception as ExceptionBase;
    if (ex.ErrorCode != ExpectedErrorCode)
    {
      string msg = string.Format("Test method {0}.{1} threw expected "
          + "exception {2} with error code {4} but error code {5} was expected.",
        base.TestContext.FullyQualifiedTestClassName, base.TestContext.TestName,
        exception.GetType().FullName, ExpectedException.FullName,
        ex.ErrorCode, ExpectedErrorCode);
      throw new Exception(msg);
    }
  }
}

This can be used like this:

[TestClass]
public class Tests
{
  [TestMethod]
  [ExpectedExceptionWithErrorCode(typeof(MessageException), 
    ExpectedErrorCode = ErrorCodes.INVALID_HEADER)]
  public void Test1()
  {
    // code which is expected to throw the exception
    // ...
  }
}