Thursday, May 23, 2013

UA Testing with Selenium and PHPUnit


Last week I spoke at php[tek] 2013 where I explained to people how to get started with Selenium IDE to record user interaction with the web interface, convert them to PHPUnit testcases and automatically execute them on multiple browsers on multiple platforms.
The feedback I got was awesome, you're all a great crowd! But on twitter I also received a bunch of questions regarding how to set up multiple platforms and why I used Windows in my presentation to deploy to.
So today I deceided it was time to write a full article on this subject.

What is Selenium?

Selenium is a tool that allows you to continuously test user interfaces of web applications. The most common usages for Selenium testing are the following:
  • testing elements are (not) on the web interface
  • users can't break out a certain flow on the web interface
  • calculated values are correct after modification
  • errors appear on screen when mistakes are made by users
  • reported issues are valid
In general we call these type of tests User Acceptance Tests or UAT and are all focused from the point of the end-user, the person using the web interface to accomplish a certain goal.

Why are they important?

UAT have their own right to exist. Just like regular unit, performance and stress tests they have their own agenda and are adressing a particular part of your application that needs testing. All to prevent your customers/visitors from finding issues, bugs or just unfunctional pieces on your web application and loose their trust in your products or services.
Therefor it's always good to invest in the "visibile" part of your web application. Especially when using javascript, you want to ensure it always works as intended.

Disclaimer

Selenium tests are in no way a replacement for regular unit tests. Their focus is on generated output of your web application within a browser. Unit tests are still necessary to ensure the logic of your application is not broken when making modifications or adding new functionality!

Setting things up

You can write your own Selenium tests by hand, but the easiest way is to use the Firefox plugin to recored Selenium tests.
Start Firefox and download the plugin at the Selenium IDE download page.
When you are asked to "restart", wait a little and scroll down on the downloads page to PHP Formatter and install this formatter plugin as well.
Once all is installed, restart your Firefox browser.
Now you're all set to get started creating UA tests.

Overview of what we're doing here

The Selenium IDE plugin allows you to record all events you make on the web application you want to test. These can be mouse clicks, entering text inside input fields, pressing enter, drag-and-drop and so much more. All activities are being captured using this recording tool.
For my example I used a simple project that's still a work-in-progress. There were already some issues reported in the bug tracker and I'm showing here how to get started with confirming the issue is truely an issue. Then we fix the issue and rerun our test to see it resolves.
Next we're going to export the Selenium test as PHPUnit test and run it as we'd normally run our unit tests. We can use our IDE, command line (as I will use) or a continuous integration (CI) system to run these tests over and over again.
Since we have a Selenium test we can modify our test case to verify our application works on multiple browsers.

Step 1: Verify an issue from bug tracker

In many cases you're already working on a project and you have issues to resolve. The easiest way to get started with UAT would be to pick an issue and verify the issue is genuine.
Issue listing
Go to your bug tracker and pick an issue. Look at the detailed description of the issue as this will be your test goal.
Detailed listing
Now that we have a full understanding of the issue, we can start recording a Selenium test. Our goal is to ensure we can reproduce the issue which is hard sometimes as reports (like this) can specify items that are missing.
Therefor we're going to add "assertions" in our Selenium test of items we expect to be present, but aren't at the moment. This may sound weird, but this way we think ahead on how we need to resolve the issue if the reported issue is genuine.

Step 2: Fix the reported issue and test again

Once we fixed the issue, we can then rerun our test and see it succeed.
We now save the Selenium test as .html file.

Step 3: Convert to PHPUnit test

In the menu of your Selenium IDE window, you can select "File" -> "Export as" -> "PHP (PHPUnit)" and store this in the application's tests directory. I have chosen to create an additional directory called "seleneum" so I can differentiate unit tests for the project.
Export as PHPUnit test

Step 5: Set up Selenium Server

In order to run your exported Selenium test in an automated, headless way you need to set up a computer that will run the Selenium Server service (as our unit tests will try to contact it for running the tests within a browser). Selenium Server is a Java daemon that will continuously listen to incomming calls from unit tests, so no matter if you run it manually from your IDE or command line, or you have a CI system to run your unit tests automatically, it should be able to contact the system.
I have chosen to use a Windows 7 64bit virtual machine as it supports the most essential browsers like Internet Explorer, Firefox, Google Chrome and others, but as the Selenium Server is a Java service, you can run it on Windows, Linux and Mac OS.
Go back to the Selenium HQ website and download the selenium-server-standalone-X.X.X.jar where X.X.X represents the current version of the tool.
Download Selenium Server
Once downloaded, you can set up your test machine for receiving Selenium Test calls. I created a simple startSeleniumStandalone.BAT file that I launch during startup.
"C:\Program Files (x64)\Java\jre6\bin\java.exe" -jar "C:\Tools\Selenium\selenium-server-standalone-X.X.X.jar" --port 12666
This will launch Selenium Server on port 12666 and will execute for each browser defined a new window.

Step 6: Modifying your PHPUnit Selenium Test

When you first export your Selenium test to a PHPUnit Testcase you need to modify a couple of things before you can execute it. This is the rawexport from Selenium IDE:
<?php
class Example extends PHPUnit_Extensions_SeleniumTestCase
{
  protected function setUp()
  {
    $this->setBrowser("*chrome");
    $this->setBrowserUrl("http://www.theialive.com/");
  }

  public function testMyTestCase()
  {
    $this->open("/");
    $this->click("link=login");
    $this->waitForPageToLoad("30000");
    $this->type("id=email", "dragonbe+tek13@gmail.com");
    $this->type("id=password", "test1234");
    $this->click("id=signin");
    $this->waitForPageToLoad("30000");
    $this->click("link=Test demo");
    $this->waitForPageToLoad("30000");
    $this->assertEquals("Done", $this->getText("xpath=//th[5]"));
    $this->click("link=[EDIT]");
    $this->waitForPageToLoad("30000");
    $this->assertTrue($this->isElementPresent("id=done"));
    $this->click("link=sign off");
    $this->waitForPageToLoad("30000");
  }
}
?>
First we need to rename our test class to something more meaningful, like MarkTaskDoneTest.
class MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase
Secondly we need to modify our setup so it can connect with our Selenium Server and run Internet Explorer.
protected function setUp()
{
    $this->setBrowser("*iexplore");
    $this->setBrowseUrl("http://www.theialive.com/");
    $this->setHost('192.168.10.33');
    $this->setPort(12666);
}
Lastly we modify our test method to something meaningful like testMarkTaskAsDone:
public function testMarkTaskAsDone()
We also add the following line to our test so on our test machine, all will be executed in full screen. Not really a required feature, but allways nice to see stuff running automatically.
$this->windowMaximize();
So our full test class would look like the following:
<?php
/**
 * Class MarkTaskDoneTest
 *
 * @group SeleniumTest
 */
class MarkTaskDoneTest extends PHPUnit_Extensions_SeleniumTestCase
{
    protected function setUp()
    {
        $this->setBrowser("*iexplore");
        $this->setBrowserUrl("http://www.theialive.com/");
        $this->setHost('192.168.10.33');
        $this->setPort(12666);
    }

    public function testMarkTaskAsDone()
    {
        $this->windowMaximize();
        $this->open("/");
        $this->click("link=login");
        $this->waitForPageToLoad("30000");
        $this->type("id=email", "dragonbe+tek13@gmail.com");
        $this->type("id=password", "test1234");
        $this->click("id=signin");
        $this->waitForPageToLoad("30000");
        $this->click("link=Test demo");
        $this->waitForPageToLoad("30000");
        $this->assertEquals("Done", $this->getText("xpath=//th[5]"));
        $this->click("link=[EDIT]");
        $this->waitForPageToLoad("30000");
        $this->assertTrue($this->isElementPresent("id=done"));
        $this->click("link=sign off");
        $this->waitForPageToLoad("30000");
    }
}
That's it! Now all is ready to run our tests using PHPUnit on the command line.
Running Selenium Test on CLI
And this is how our unit tests run on our test machine:

Step 7: Testing with multiple browsers

Running these tests is fun, but they still test a single browser. A better way would be to set up a base TestCase where you set all browser configurations and extend this base TestCase whenever you create a Selenium Test.
File: tests/SeleniumTestCase.php
<?php

require_once 'PHPUnit/Extensions/SeleniumTestCase.php';

class SeleniumTestCase extends PHPUnit_Extensions_SeleniumTestCase
{
    const TEST_HUB = '192.168.10.33';
    const TEST_PORT = 12666;

    const USERNAME = 'dragonbe+tek13@gmail.com';
    const PASSWORD = 'test1234';
    const BASURL = 'http://www.theialive.com';

    public static $browsers = array (
        array (
            'name' => 'Internet Explorer 8 on Windows 7',
            'browser' => '*iexplore',
            'host' => self::TEST_HUB,
            'port' => self::TEST_PORT,
        ),
        array (
            'name' => 'Firefox on Windows 7',
            'browser' => '*firefox',
            'host' => self::TEST_HUB,
            'port' => self::TEST_PORT,
        ),
        array (
            'name' => 'Google Chrome on Windows 7',
            'browser' => '*googlechrome',
            'host' => self::TEST_HUB,
            'port' => self::TEST_PORT,
        ),
    );

    protected function setUp()
    {
        $this->setBrowserUrl(self::BASURL);
    }
}
Now we just need to extend from our base test case to continuously use all configured browsers.
File: tests/selenium/TaskTest.php
<?php
/**
 * Class MarkTaskDoneTest
 *
 * @group SeleniumTest
 */
require_once 'SeleniumTestCase.php';
class MarkTaskDoneTest extends SeleniumTestCase
{
    public function testMarkTestAsDone()
    {
        $this->windowMaximize();
        $this->open("/");
        $this->click("link=login");
        $this->waitForPageToLoad("30000");
        $this->type("id=email", TestCase::USERNAME);
        $this->type("id=password", TestCase::PASSWORD);
        $this->click("id=signin");
        $this->waitForPageToLoad("30000");
        $this->click("link=Test demo");
        $this->waitForPageToLoad("30000");
        $this->assertEquals("Done", $this->getText("xpath=//th[5]"));
        $this->click("link=[EDIT]");
        $this->waitForPageToLoad("30000");
        $this->assertTrue($this->isElementPresent("id=done"));
        $this->click("link=sign off");
        $this->waitForPageToLoad("30000");
    }
}
Running it on CLI will execute 3 times this test as we're testing 3 browsers: Internet Explorer, Firefox and Google Chrome.
Running on CLI
On our test machine it would look something like this:

Closing Remarks

Authentication

When running Selenium Tests and your application requires authentication, the best thing you can do is to have your test do the following:
  • log into your application (carefull, passwords are plain text!!!)
  • run your tests and assertsions
  • log out of your application
This way your tests always behave the same.

Multi platform setup

As good developer I not only test on different browsers, but also on multiple platforms. I use virtual machines for this that are always running and I use the following setup to achieve this:
Multi-node testing with Selenium

Dom changes

Be careful as Selenium tests heavily depend on the DOM of a web application. Any changes there can have an effect on your Selenium Tests and might result in failing tests.

Recommended reading

See my presentation as it was given at php[tek] 2013
These books I can highly recommend reading!
   
The Grumpy Programmer's PHPUnit Cookbook
The Grumpy Programmer's Guide To Building Testable PHP Applications

Monday, May 20, 2013

Survived php tek 2013

If you were last week in Chicago, you've might felt the city was buzzing PHP all over the place. php[tek] 2013 was taking place at the Sheraton Gateway Suites Chicago O'Hare in Rosemont, just outside of Chicago city.

This year it was also the first time Musketeers.me, a php consulting team from the East Coast, was running the show, putting their own signature onto the event. And with great success I might add. A well deserved applause to Eli White, Kevin Bruce, Sandy Smith, Oscar Merida and of course the Beth Tucker Long for their unlocked achievement running a great conference.

For me was also the first time I was running the uncon, where attendees and conference speakers could propose talks they wanted to present at the uncon and have other attendees vote for them.

One uncon talk really stood out: "Open Sourcing mental illness" by Ed Finkler (@funkatron) where he discussed the issues people face who suffer a mental illness. It was an emotional talk where Ed described his own experiences and how his mind made him think about things differently then the rest of us. For me it was a real eye-opener and made me understand that there are people that don't really take things for granted.

This year's edition was filed with very good talks and it's almost impossible to give my feedback on all of them. A few talks that I attended really stood out that I would really want to promote here.

Chris Cornut, the driving force behind phpdeveloper.org, talked about "Beyond the Basics: Security with PHP" where he did not just list the top 10 of OWASP, but also gave good advices on how to protect yourself against most of the common attacks. A good closing hint: be better secured than the next guy.

Jeremy Kendall was giving good advice on how to improve your code you wrote a couple of weeks or months ago in his talk "PHP 102: Out with the Bad, In with the Good". What I like about Jeremy is he can bring very complex subjects in an easy, understandable way that even a novice can understand complex software engineering stuff. And so he did with this talk.

One talk I missed which I really wanted to see was the Distractions talk of Sean Prunka, or how to deal with distractions when you're a developer. When I look at the reviews on joind.in I see he did an amazing good job giving this was his first conference talk ever. So hopefully he will do a webinar or an online recording of this talk *hint, hint*.

This year my company was sponsoring the hackathon and can be called a good success, knowing we had to compete against Lego fun party. According to Lorna a bunch of pull requests were made for joind.in (the community feedback platform for conference speakers). And we captured the first pull request on twitter.


At the introduction of the hackathon we also had a nice surprise for Mr. Keith Casey from his colleagues at Twillio where he worked 2 years now.


And of course the hallway tracks were not only fun, but also hugely valuable for the community. A spontanious group were giving lessons on doing TDD, others were hacking on gadgets, or just having discussions on best practices. Guess who talked about "how to get started with a user group" and got 3 people stating they would either start or reboot a user group in their area. So Riga in Latvia, Charlottesville in Virginia and Paris in France: get ready as there's a php user group coming near you!

In my experience the best php[tek] ever, and I'm really looking forward to the 2014 edition. If you don't believe me, have a look at the pictures taken at php[tek] 2013. They will tell the story.


Getting to 100% code coverage Sitting at @ramseyben his talk "API first" #tek13 Teenage Mutant PHP Turtles taking over @rdohms ' talk #tek13 Hackathon now #tek13 Listening to @jeremykendall about better coding #tek13


Maybe I'll see you next year at #tek14!



Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License.