I have a test where I want to ensure separate operations within one page result in distinct results. Specifically, I have a few ways to sort on a page and I want a test to make sure that each sort is different. I have other tests to ensure the correctness of each sort.
I would like the focus of this conversation to be on a good way to run test operations in parallel and compare the results at the end, rather than on what to test or testing methods. I figure parallel operations in testing is an interesting and broad enough topic that it could be useful to others.
Let "generateHashFromSearchResults()" be a function that returns a string representing the order of the search results shown on current IE instance. Here is what the working code looks like in a serialized fashion using one browser instance:
var set = new HashSet<string>();
var sortOptions = new List<String>() { "sort1", "sort2", "sort3" };
// Default sort
set.Add(generateHashFromSearchResults());
sortOptions.ForEach(s => {
ie.Link(Find.ByText(s)).Click();
set.Add(generateHashFromSearchResults());
});
Assert.That(set.Count() == 4);
I had read about PLINQ a few months ago and figured this might be a decent use case. Now let "generateHashFromSearchResults(IE ie)" be the same function, but that operates on an explicitly defined IE instance. I tried something like this:
List<string> resultList = sortOptions.AsParallel().Select(s => {
var ie = new IE(true);
ie.Link(Find.ByText(s)).Click();
return generateHashFromSearchResults(ie);
}).ToList();
// Forget about default sort for now. There should be 3 distinct results
Assert.That(new HashSet<string>(resultList).Count() == 3);
The biggest issue I face right now is not understanding how PLINQ does thread management. WatiN needs to run with the apartment state set to single threaded (STAThread). I get that each IE instance should be in its own thread, but no amount of setting each thread in the PLINQ query to the proper apartment state fixes the issue.
I'm starting to suspect that I either need to learn more about PLINQ to continue, or that I need to learn more about thread management by hand to get this to work.
Any thoughts?
You can't specify a custom scheduler with AsParallel(). But you can create a Task for each sort option and pass an instance of a custom scheduler into the Start() method. This implementation of an STA Thread scheduler was borrowed from Stephen Toub (http://blogs.msdn.com/b/pfxteam/archive/2010/04/07/9990421.aspx):
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
/// <summary>
/// Provides a scheduler that uses STA threads.
/// Borrowed from Stephen Toub's implementation http://blogs.msdn.com/b/pfxteam/archive/2010/04/07/9990421.aspx
/// </summary>
public sealed class StaTaskScheduler : TaskScheduler, IDisposable
{
/// <summary>
/// The STA threads used by the scheduler.
/// </summary>
private readonly List<Thread> threads;
/// <summary>
/// Stores the queued tasks to be executed by our pool of STA threads.
/// </summary>
private BlockingCollection<Task> tasks;
/// <summary>
/// Initializes a new instance of the StaTaskScheduler class with the specified concurrency level.
/// </summary>
/// <param name = "numberOfThreads">The number of threads that should be created and used by this scheduler.</param>
public StaTaskScheduler(int numberOfThreads)
{
if (numberOfThreads < 1)
{
throw new ArgumentOutOfRangeException(
"numberOfThreads", "The scheduler must create at least one thread");
}
// Initialize the tasks collection
this.tasks = new BlockingCollection<Task>();
// Create the threads to be used by this scheduler
this.threads = Enumerable.Range(0, numberOfThreads).Select(
i =>
{
var thread = new Thread(
() =>
{
// Continually get the next task and try to execute it.
// This will continue until the scheduler is disposed and no more tasks remain.
foreach (Task t in this.tasks.GetConsumingEnumerable())
{
this.TryExecuteTask(t);
}
}) {
Name = "Sta Thread", IsBackground = true
};
thread.SetApartmentState(ApartmentState.STA);
return thread;
}).ToList();
// Start all of the threads
this.threads.ForEach(t => t.Start());
}
/// <summary>
/// Gets the maximum concurrency level supported by this scheduler.
/// </summary>
public override int MaximumConcurrencyLevel
{
get
{
return this.threads.Count;
}
}
/// <summary>
/// Cleans up the scheduler by indicating that no more tasks will be queued.
/// This method blocks until all threads successfully shutdown.
/// </summary>
public void Dispose()
{
if (this.tasks != null)
{
// Indicate that no new tasks will be coming in
this.tasks.CompleteAdding();
// Wait for all threads to finish processing tasks
foreach (Thread thread in this.threads)
{
thread.Join();
}
// Cleanup
this.tasks.Dispose();
this.tasks = null;
}
}
/// <summary>
/// Provides a list of the scheduled tasks for the debugger to consume.
/// </summary>
/// <returns>An enumerable of all tasks currently scheduled.</returns>
protected override IEnumerable<Task> GetScheduledTasks()
{
// Serialize the contents of the blocking collection of tasks for the debugger
return this.tasks.ToArray();
}
/// <summary>
/// Queues a Task to be executed by this scheduler.
/// </summary>
/// <param name = "task">The task to be executed.</param>
protected override void QueueTask(Task task)
{
// Push it into the blocking collection of tasks
this.tasks.Add(task);
}
/// <summary>
/// Determines whether a Task may be inlined.
/// </summary>
/// <param name = "task">The task to be executed.</param>
/// <param name = "taskWasPreviouslyQueued">Whether the task was previously queued.</param>
/// <returns>true if the task was successfully inlined; otherwise, false.</returns>
protected override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
// Try to inline if the current thread is STA
return Thread.CurrentThread.GetApartmentState() == ApartmentState.STA && this.TryExecuteTask(task);
}
}