I’ve been learning new things again (unfortunately my MongoDB efforts have been put aside yet again). One of the cool new things that I’ve heard of in passing, but never got curious about was Selenium, a tool that automates actions in the browser. I have had a lot of fun playing around with both the IDE and WebDriver. Unfortunately, I hit some snags when my lead requested I try out Selenium with SpecFlow, a behavior testing framework.

One of the frustrating hurdles with using so many tools, libraries, and frameworks is the documentation is often not there. I have to get creative with my search queries and it’s taxing on top of getting whatever I need to get done.

One example. I will, at some point, need to convert the XML output from Selenium/NUnit to a more readable format. A parallel is MSTest’s TRX file to HTML, which someone wrote an excellent tool for (trx2html) and the formatting gets the information across in a simple format. There were numerous other tools to make HTML formats from NUnit results that would display nicely on this or that resolution, but really, I just needed rows and whether the tests passed or not, not boxes.

Someone did write a tool to convert to an HTML format which looks similar to trx2html. Unfortunately, this relied on NAnt (software automation), which I did not use nor need. I did manage to find the non-automated tool but it was severely lacking in documentation on how to just use it–I didn’t want to download it just to run “/?” for instructions, but that’s what I ended up doing. trx2html is also guilty of this, but someone did post online the basic use of it…it was just buried.

The above example is fairly trivial, but why are we so lazy to explain how to use our own software in a most basic use-case scenario?

After finally figuring out the basics and good practices (Page Object Model) of Selenium WebDriver and many search queries later, I had a decent set of tests when my lead sprung on me to take a look at using SpecFlow. Not wanting to throw away my work, I ported my tests over to the new project and stuck it under it’s own folder. I made a separate folder for SpecFlow tests. Good thing, too.

SpecFlow is the most frustrating thing I’ve had to work with lately. I thought it would be straightforward, just a little more tedious. Write test cases in Gherkin, use the nifty tool to auto-generate the methods, fill in the methods with code from my original tests, and done. This worked for the first test. Then I went on to make the next test and that’s when things went sideways.

I have two pages, one registers new users and one edits existing users. As you can imagine, many of these fields are the same (e.g., username, password, name, email). My base page contains these shared fields and the specific pages contain whatever is missing respectively.

In SpecFlow, the methods that are generated are global methods. What this means is the following. Suppose I have the following scenario in new user:

Given ...
When I press Submit
Then ...

In my step file (actual test code side), I have:

// Other code
[When(@"I press Submit")]
public void WhenIPressSubmit()
{
	_newUserPage.Submit();
}
// More code

This will cause problems if I use the same Gherkin line in my scenario concerning editing a user. Because I don’t want to use the _newUserPage object, but _editUserPage.

I thought the fix would be scoping the methods to their respective feature, but apparently that’s an anti-pattern. In other words, a 1:1 mapping between features and step methods/classes is a no-no. Why? Because it’s “evil” and the preference is for these steps to be reusable across features and scenarios (look it up on their wiki). This just sounds like they’re trying to apply OOP but frowning on the proper scoping that comes with namespaces, classes, and inheritance.

So, I guess I have to fine tune my text with “When I press Submit to create a new user”, which makes it more specific to the context in which I want to use it. I suppose those who are more verbose (e.g., business) like this sort of thing.

Oh, and don’t get me started on the duplication of code between Given and When methods concerning buttons, either.

Meanwhile, my SpecFlow tests are currently a mess while I sort this out.

The other thing about SpecFlow’s global bindings is setting up and tearing down scenarios. I wasn’t aware of this (it wasn’t immediately obvious from the basic getting-started examples) and ended up with 5GB of wasted space in my %temp% folder. How did this happen?

Selenium WebDriver creates temp folders in %temp% when it’s run and upon a proper Quit(), will clean up after itself. I noticed, for some reason, my SpecFlow tests were popping up two windows. I realized this was because I had defined two setup methods in two different step files, thinking they only applied to their respective feature. It was actually running both setup methods, but since I was only testing one at a time, one browser would just sit, twiddle, and exist past it’s expiration while the other executed the test and closed properly.

I’m sure there’s some method of thinking to coding SpecFlow tests that I haven’t quite figured out, but right now it’s just frustrating and illogical. In short, I’m pretty happy with Selenium WebDriver, not so much with SpecFlow.