Silverlight Unit and Integration Testing

July 2, 2009

I’m going to tell you that, with a bit of effort, you can get a really good Test Driven Red/Green cycle with Silverlight…

Assumptions – you have installed VS 2008, and the Silverlight 2 SDK

Firstly – we created 4 projects in Visual Studio:

  • Silverlight.App – holds the production code
  • Silverlight.App.UnitTests – runs tests that require no external dependencies
  • Silverlight.App.IntegrationTests – contains tests that connection to the web, database, are asynchronous, etc
  • Silverlight.App.IntegrationHost – a ASP.NET project that will run the IntegrationTests

UnitTests and IntegrationTests are created with the Silveright UnitTesting project templates. These templates cause a test page to be created. The test page basically listens to events from the TestRunner (embedded in the Test silverlight applications), and presents it as (fiddly) HTML.

Our UnitTest project hasn’t changed much since we started. We have had issues with the IntegrationTests project:

Security

  • Code transparency – All application logic is marked as “Transparent”, which means that may call but may not extend or override “SafeCritical” or “CriticalCode”. The System.Net.WebClient is marked as SafeCritical. This means that you won’t be able to Mock/Stub WebClient. At all.
  • Internal event constructors – The constructor for DownloadStringAsyncCompleteEventArgs is internal; You won’t be able to simulate a WebClient completing an http request.
  • Silverlight applications loaded from the file system just plain cannot connect to web resources. There is no security policy that will allow it.

All of the above issues drove us to use the IntegrationHost to run the IntegrationTests, but in a web context. This is quite good, especially for within-IDE testing, since you can host your test data as flat files or as ASP driven test data in the same host.

Unit Testing

Good news

  • Rhino mocks is available for Silverlight.
  • Nunit is available as source for Silverlight (with some shims for older, pre-silverlight data structures).
  • The Silverlight UT runner works with both MSTest Metadata and nunit style assertions at the same time

Puzzling

  • Nunit lite appears to be at version 0.5, and maybe worth looking at instead of a custom build of Nunit
  • Specificity Looks like it may work in the Silverlight environment.
  • Asynchronous testing using the [Asynchronous] attribute is potentially very brittle
  • Maintaining the test data for the integration tests in the IntegrationHost web project seems counter intuitive

Continous Integration

  • Silverlight UT test reports are shockingly bad HTML, making downstream consumption particularly difficult. Would be nice to log semantic html somewhere.
  • For the unit tests, we found a powershell script that will spawn IE and scrape the results into a file
  • For the integration tests, we modified the powershell script to spawn a WebDev.WebServer.exe against the IntegrationHost
  • The IntegrationTest xap file that is linked from the IntegrationHost will not get automatically deployed by msbuild – we use a copy task to pull it before launching IE.
  • Visual Studio wants to put the hosted xap file in ClientBin of the IntegrationHost. This is fine, but it also wants to put it into source control, which is not. Eventually, you’ll get the right combination of check-ins, deletes, et. al. and it won’t be an issue. Expect to lose a few hours on this.

Powershell

When you have to clear out some space for that web server socket:

gwmi win32_process | where { $_.CommandLine -like "*WebDev.WebServer.exe*/port:$port" } | Kill

Asynchrony

Async silverlight tests look a bit like this:

[Asynchronous]
public void ShouldHangAroundABitWaiting {
   EnqueueCondition( () => return TrueIfCanStart() );
   EnqueueCallback( () => Specify.That( something, Is.Ready() );
   EnqueueTestComplete();
}

It is at times like this that one definitely wants a monad.

One day, async code will look like this:

[Asynchronous]
public void ShouldHangAroundABit {
	TrueIfCanStart.Wait();
	Specify.That( something, IsReady() )
}