Silverlight: Unit Testing

Overview

Microsoft has made their internal Silverlight unit testing framework available for all developers. It provides an extensible testing solution for applications, controls and class libraries. This page goes over the basics of unit testing for Silverlight applications and how to set up test projects, but does not go over unit testing methodology.

Getting Started

To get started, you must first download the Silverlight Unit Test Framework. You can also download the Visual Studio 2008 project templates for Visual Basic and C#.

Set Up the Development Environment

  1. Download Microsoft.Silverlight.Testing.zip.
  2. Extract and copy the binaries to %PROGRAMFILES%\Microsoft SDKs\Silverlight\v2.0\Libraries\Client.
  3. Download Visual Studio Templates.zip.
  4. Extract and copy the project template ZIP archives to the corresponding language directories in
    %USERPROFILE%\[My] Documents\Visual Studio 2008\Templates\ProjectTemplates.
  5. Extract and copy the item template ZIP archives to the corresponding language directories in
    %USERPROFILE%\[My] Documents\Visual Studio 2008\Templates\ItemTemplates.

The source code for the testing framework is available as part of the Silverlight Toolkit on CodePlex.

Start a New Test Project

Test projects are commonly added to the same solution where your main application or library resides. This makes it easier to reference the project with the classes being tested and also helps remind that unit testing is a core task before and during development.

  1. Start Visual Studio 2008.
  2. Open the New Project dialog.
  3. Click on the desired language Visual C# or Visual Basic.
  4. Select Silverlight Test Project under the My Templates heading.
  5. Enter the requested project and solution information, and then click OK.

At this point it is recommended to verify that the project namespace matches your naming conventions. A common naming pattern is to use your main project's namespace appended with Test or Testing. For example if your main project has the namespace Yahoo.Samples.Silverlight, the namespace for the tests would be Yahoo.Samples.Silverlight.Testing.

  1. Rename the namespace for the auto-generated files. Select Refactor, and then click Rename... over the namespace in the Test.cs file.
  2. Open the properties for the newly created test project.
  3. On the Silverlight tab, verify that the Assembly name matches your naming conventions. Default namespace should have been updated during the refactoring.
  4. Make sure to update the Startup object to match the new namespace of the application.
  5. Close the project properties.

Finally, add a project reference to each project you are testing.

Running Tests

At this point you should have a test project created with a stub test class and references to the projects you are testing. To verify everything was set up correctly, you can simply run the test project as you would any other project. However, running the test project in debug mode will result in Visual Studio breaking into debug mode for every inconclusive and failed test, along with every expected exception. The avoid this you have several options:

  • Run your tests without debugging (Ctrl-F5 / Start without Debugging). However, this will make it harder to debug bugs in the tests themselves.
  • Disable breaks for unhandled user exceptions for the various unit testing exceptions in the Debug/Exceptions... dialog. However, this still does not solve breaking for expected exceptions unless you use try/catch block around the expected exceptions and manually fail if the exception was not raised. Add the following exceptions and uncheck the User-unhandled check for them:
    • Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException
    • Microsoft.VisualStudio.TestTools.UnitTesting.AssertInconclusiveException
  • Disable breaks for unhandled user exceptions for all CLR Runtime Exceptions in the Debug/Exceptions... dialog. However, this affects all projects which is usually not desirable.

Unit Test Basics

Writing unit tests is similar to writing the actual code for your project. In both cases you create a project and classes that use various objects. As such, you should take the same care in writing and documenting your tests as you do with the actual code. The main difference is that test classes and methods are marked with special attributes to inform the testing framework of various aspects about the code and provide contextual information in the test results.

Common Test Attributes

The following table lists the most common attributes used when writing tests.

Attribute Description
TestClass Specifies that the class contains methods to run during a test.
TestMethod Specifies that the method is a test.
ExpectedException(exceptionType, message) The given exception should happen during the test for the test to be considered successful.
Asynchronous The test has completed only after the TestComplete method is called, not when the method exits. Your test class must inherit from SilverlightTest to perform asynchronous tests.
Tag(description) Marks a test as belonging to the named group. Multiple tags can be applied to a single class/method. Tags are also automatically generated with names in the format "namespace + name".
Description(description) Describes the test to give more context when viewing the test results.
Bug(Description, fixed) Describes a known issue and flips the test result.

Assertions

Assert class methods are used to compare values in a test against the expected results. The following table lists the most common methods.

Method Description
AreEqual / AreNotEqual Verifies that the given values are/are not equal.
AreSame / AreNotSame Verifies that the given variables refer to the same / different objects.
Fail Fails the test.
Inconclusive Test was inconclusive. Often used to mark that the test hasn't been implemented yet.
IsFalse / IsTrue Verifies the condition is false / true.
IsInstanceOfType / IsNotInstanceOfType Verifies the given object is / is not of a specified type.
IsNull / IsNotNull Verifies the given object is / is not null (Nothing in VB).

Example

The following example shows a basic test class with a simple test method. This test will fail since it has been marked with the Bug attribute with the Fixed property set to false. The bug would then be fixed later and the Fixed property set to true and the test would pass.

C# Example
Visual Basic Example

Asynchronous Tests

The test framework has basic support for testing asynchronous scenarios such as events, UI tasks and multi-threaded operations. Performing asynchronous tests involves pushing tasks into a queue and calling the TestComplete method to notify the test framework that the test has completed.

Example

The following shows an example of testing for an asynchronous event. A timer is created to wait two seconds and an event handler is hooked up to signal that the event was raised. Tasks are then queued up to trigger the event, In this case tasks include starting the timer, waiting for the event to be raised, and finally checking an additional value that should have been set.

NOTE: Currently the Timeout attribute is not implemented and the test runner will hang if the event isn't raised. You may wish to add a custom timer to avoid this situation, especially if you are running tests as part of an automated process.

C# Example

Tips

Here are some general tips for writing tests:

  • Consider unit test code to be "real" code. Follow standard coding and naming conventions and comment your code.
  • A single test should normally test one thing. Don't be afraid of having many tests.
  • Test to see if your code validates any input it receives. This will make your code base more robust and secure.
  • Take extra care to validate for boundary conditions. If a method accepts a number between a certain range, make sure to test for the minimum and maximum values and also values just above and below the allowed values.
  • Even if you aren't following test driven development, think about how to test your classes while architecting and writing your code. This will almost always result in better designed software.
  • Each class to test should have a corresponding test class. If you feel like your test class is "too large", it may be an indicator that your main class structure requires refactoring.
  • Automate tests whenever possible. Create a script to run the tests daily or in conjunction with check-ins to a source control system.
  • Consider code coverage tools to help you fill in any missing tests.

Further Reading

Related information on the web is listed below.