Why automated browser testing should be automated

That was an interesting thing to see. I had just deployed the compatibility tester to the production site, tested it, gone to bed, and saw that tweet in the morning.

“Whaaaat?” I thought to myself? I have automated tests, and I’m building an automated testing framework. Not only that but I have automated tests on that functionality. How did I miss this?

Quite easily, actually. The Ajax link for sending the email was built when I first was working on the feature. But in between then and my deployment I had changed the routing mechanism for doing that. And since it had already been tested (when I first started) I didn’t think of testing it again. Why? Because it was tested already. I hadn’t thought of testing the email form. It was a minor thing, after all.

Silly me. Thankfully fixing the route was a simple solution.

Takeaways

You may not have a full test suite for your site. In fact, it is possibly on the verge of impossible to test every single problem.

But that doesn’t mean you can’t start.

And where should you start?

Start with bugs

  1. Receive a bug report
  2. Replicate the bug manually
  3. Automate bug replication using Magium
  4. Fix bug
  5. Confirm fix using the test case

Test change

The easiest way to replicate the bug was to click the submit button and check for an alert since that would be an unexpected result since it were a positive test. But if I fixed the bug then that condition should not occur. Instead I added the following lines of code to the test.


$this->webdriver->byId('my-email-address')->clear()->sendKeys('[email protected]');
$this->webdriver->byId('submit-email-address')->click();
$this->sleep('1s');
$this->byXpath('//body');

That replicates the issuebecause byXpath() will trigger an error if there is an alert() present, but will return the <body> element if it’s fine.