Posts Tagged ‘testing’

posted on Friday 4th March 2011 by Dave

Mocking Iterator with PHPUnit

When writing unit tests, it is important that you isolate the system under test from any dependencies. Put simply, this means you have to mock any other objects that your system under test interacts with. This can be tricky when a dependency implements the Iterator interface as you have to carefully mock all calls to five methods in the correct order.

This blog post provides an example test showing how to mock an Iterator in PHPUnit as well as a helper class to make this process straightforward. You can view the source code for the helper on PasteBin or download the entire source code as a gzipped TAR.

Simple Iterator class

To demonstrate, consider the following simple iterator class.

/**
 * Example list class
 *
 * @author Dave Gardner <dave@davegardner.me.uk>
 */
class exampleList implements Iterator
{
    /**
     * Our items
     *
     * @var array
     */
    private $items = array(
        'item1' => 'This is the first item',
        'item2' => 'This is the second item',
        'item3' => 'This is the third item',
        'item4' => 'This is the fourth item',
        'item5' => 'This is the fifth item'
    );

    /**
     * Get key of current item as string
     *
     * @return string
     */
    public function key()
    {
        echo "\033[36m" . __METHOD__ . "\033[0m\n";
        return key($this->items);
    }

    /**
     * Test if current item valid
     *
     * @return boolean
     */
    public function valid()
    {
        echo "\033[36m" . __METHOD__ . "\033[0m\n";
        return current($this->items) === FALSE
                ? FALSE
                : TRUE;
    }

    /**
     * Fetch current value
     *
     * @return string
     */
    public function current()
    {
        echo "\033[36m" . __METHOD__ . "\033[0m\n";
        return current($this->items);
    }

    /**
     * Go to next item
     */
    public function next()
    {
        echo "\033[36m" . __METHOD__ . "\033[0m\n";
        next($this->items);
    }

    /**
     * Rewind to start
     */
    public function rewind()
    {
        echo "\033[36m" . __METHOD__ . "\033[0m\n";
        reset($this->items);
    }
}

Ignore the strange bash escape codes (I can't help myself when writing CLI scripts). Instantiated objects of this class will loop through their five internal items (when asked) and echo out the methods being called in cyan! We can see this in action via the following code:

echo "\n\033[44;37;01mTest 1: foreach key => value\033[0m\n\n";

$list = new exampleList();
foreach ($list as $key => $value)
{
    echo "\033[01m$key = $value\033[0m\n";
}

echo "\n\033[44;37;01mTest 2: foreach value\033[0m\n\n";

$list = new exampleList();
foreach ($list as $value)
{
    echo "\033[01m$value\033[0m\n";
}

When we run this, we get the following.

Running the iterator

Mocking an Iterator for testing

This shows us exactly which methods are called and in what order. We can now use this to write some tests for a mocked iterator. All we have to do with our mocked object is set up PHPUnit expectations; the key point being the use of the at() matcher to specify the exact sequence of calls. The following test applies this to allow us to iterate through three mocked items. You can view the full source on PasteBin.

    public function testWhenMockThreeIterationWithNoKey()
    {
        $list = $this->buildSystemUnderTest();

        $list->expects($this->at(0))
             ->method('rewind');

        // iteration 1
        $list->expects($this->at(1))
             ->method('valid')
             ->will($this->returnValue(TRUE));
        $list->expects($this->at(2))
             ->method('current')
             ->will($this->returnValue('This is the first item'));
        $list->expects($this->at(3))
             ->method('next');

        // iteration 2
        $list->expects($this->at(4))
             ->method('valid')
             ->will($this->returnValue(TRUE));
        $list->expects($this->at(5))
             ->method('current')
             ->will($this->returnValue('This is the second item'));
        $list->expects($this->at(6))
             ->method('next');

        // iteration 2
        $list->expects($this->at(7))
             ->method('valid')
             ->will($this->returnValue(TRUE));
        $list->expects($this->at(8))
             ->method('current')
             ->will($this->returnValue('And the final item'));
        $list->expects($this->at(9))
             ->method('next');

        $list->expects($this->at(10))
             ->method('valid')
             ->will($this->returnValue(FALSE));

        $counter = 0;
        $values = array();
        foreach ($list as $value)
        {
            $values[] = $value;
            $counter++;
        }
        $this->assertEquals(3, $counter);

        $expectedValues = array(
            'This is the first item',
            'This is the second item',
            'And the final item'
        );
        $this->assertEquals($expectedValues, $values);
    }

Making this process simple via a helper

There a bunch of other tests within the full source code; I’ve left them out here to spare you a huge code block! This is actually a win. We have mocked an iterator and it works as expected. The only downside is that there’s a lot of stuff to repeat each time. To avoid this we can simply make a helper method to do the job for us. You can view the source code for this on PasteBin.

    /**
     * Mock iterator
     *
     * This attaches all the required expectations in the right order so that
     * our iterator will act like an iterator!
     *
     * @param Iterator $iterator The iterator object; this is what we attach
     *      all the expectations to
     * @param array An array of items that we will mock up, we will use the
     *      keys (if needed) and values of this array to return
     * @param boolean $includeCallsToKey Whether we want to mock up the calls
     *      to "key"; only needed if you are doing foreach ($foo as $k => $v)
     *      as opposed to foreach ($foo as $v)
     */
    private function mockIterator(
            Iterator $iterator,
            array $items,
            $includeCallsToKey = FALSE
            )
    {
        $iterator->expects($this->at(0))
                 ->method('rewind');
        $counter = 1;
        foreach ($items as $k => $v)
        {
            $iterator->expects($this->at($counter++))
                     ->method('valid')
                     ->will($this->returnValue(TRUE));
            $iterator->expects($this->at($counter++))
                     ->method('current')
                     ->will($this->returnValue($v));
            if ($includeCallsToKey)
            {
                $iterator->expects($this->at($counter++))
                         ->method('key')
                         ->will($this->returnValue($k));
            }
            $iterator->expects($this->at($counter++))
                     ->method('next');
        }
        $iterator->expects($this->at($counter))
                 ->method('valid')
                 ->will($this->returnValue(FALSE));
    }

Now we can repeat our test using the more succinct:

    public function testWhenMockThreeIterationWithNoKey()
    {
        $list = $this->buildSystemUnderTest();

        $expectedValues = array(
            'This is the first item',
            'This is the second item',
            'And the final item'
        );
        $this->mockIterator($list, $expectedValues);

        $counter = 0;
        $values = array();
        foreach ($list as $value)
        {
            $values[] = $value;
            $counter++;
        }
        $this->assertEquals(3, $counter);

        $this->assertEquals($expectedValues, $values);
    }
posted on Thursday 23rd September 2010 by Dave

Effective load testing with Apache JMeter

Load testing is surely one of the most important activities that many developers ignore. I would include myself in that bracket; it is far too often something that gets bounced out of a busy schedule. However load testing, and its cousin stress testing, are absolutely essential when attempting to create a reliable application.

This blog post concentrates on load testing, which Wikipedia defines as:

Load testing is the process of putting demand on a system or device and measuring its response.”

This is subtly different to stress testing, which aims to test a system “beyond normal operational capacity, often to a breaking point”. Load testing is useful to ensure that your application meets the business requirements, for example “cope with x million page unique users per day with a response time of less than 1 second”.

This post covers:

Things to consider when load testing

It’s very easy to carry out useless load testing. I know this from experience. We were testing a system designed to serve 30 API requests per second, each response containing data on a user from a pool of roughly 100,000,000. When we carried out our load testing we designed a test using a pool of 1,000 unique user IDs. Spot the obvious problem! Testing with only 1,000 unique user IDs meant that we quickly reached a point where various layers of caching would be happily dealing with the job of fetching this limited set of data. This includes MySQL’s key cache plus the operating system file system cache.

To avoid making the same mistakes, make sure you consider the following points when load testing.

  1. Use indicative hardware
    When load testing to ensure that an application reaches minimum performance standards, you will need to test on hardware that mirrors your “live” setup as closely as possible. Sometimes this is easy, for example if you are running a live system consisting of three servers. If however you are running a 30 server cluster including a 100TB database, getting an accurate staging system can be hard work.
  2. Use indicative data
    If you have a production system with 100,000,000 rows of data, it’s a good idea not to carry out load testing on a staging system with only 300 rows. Similarly, if your application stores rows that are roughly 100KB each, the load testing system should be designed to replicate these conditions. The closer you can get to your actual data, the more useful the results.
  3. Think about system state
    Layers of caching can make load testing hard work. This includes application caching such as Memcache plus operating system caching and database caching. If your live environment workload can expect warmed caches then it is valid to leave these things warmed! However running a load test over and over again may get your application into an unnatural state of “preparedness”.

Using Apache JMeter

It may be tempting to write your own load test tools. Why not indeed? A few CURLs, a few processes; simple. My advice would be this: before you do this, try out JMeter. You can download JMeter from this page.

This example job reads in a list of user UUIDs from a text file and then makes an HTTP request for each user.

1. Launch JMeter

sh bin/jmeter.sh &

2. Configure job

  • Right click on Test Plan heading, click Add > Thread Group
  • Right click on Thread Group, click Add > CSV Data Set Config
  • Click on the newly added config element and enter a valid Filename – this should contain one user UUID per line
  • Enter userId under Variable Names (comma-delimited)
  • Click on the Thread Group, configure the Number of Threads (say 5), the Ramp-Up Period (say 180) and then the test Duration (say 600). This tells JMeter to launch up to 5 threads, ramping these up over 2 minutes, with the test running for a total of 10 minutes.

Thread group settings

  • Right click on Thread Group, click Add > Sampler > HTTP Request
  • Click on the newly added sampler element and enter a valid Server Name or IP
  • You should probably add in some Timeouts; depending on your desired performance under load
  • Enter a Path, including the previously defined variable as follows: /1/user/${userId}/data.json

Screen shot 2010-09-23 at 14.41.36

3. Add a listener

This was a stage that took me a while to figure out! You must have at least one listener, otherwise you cannot view any results from the job. You can add listeners to the Thread Group in the same way you added samplers and config elements. The Summary Report is very useful, as is View Results Tree (especially during debugging).

4. Run the job!

It’s usually a good idea to Clear All (from the Run menu) before you fire off the job.

5. Profit

Epic Fail!

Verifying the details of a response

By default JMeter will work with HTTP response codes – which is handy. So 404 errors and 500 errors etc… will all be dealt with. It’s often useful to look at the actual content of the response to decide on success. This can help catch things like PHP Fatal Errors (where you may have a 200 OK response, but a completely blank document). This is where the Response Assertion comes in.

Right click on any sampler and then Add > Assertion > Response Assertion. You can then define patterns to test against; these can use regular expressions. One other point to note is that if you wish to have HTTP 500 or 400 errors ignored (treated as success) then you should tick the Ignore Status tick box.

Extracting data from a response

Hitting URLs with known UUIDs is a handy tool, however it’s also useful to call a URL to add a user, extract the newly added user UUID and then use this in any subsequent requests. Fortunately this is also very easy to achieve with JMeter, using the Regular Expression Extractor.

Right click on any sampler and then Add > Post Processors > Regular Expression Extractor. The following settings extract data from a JSONP response which includes a userId definition; this is then assigned to the JMeter variable userId.

  • Reference Name: userId
  • Regular Expression: var userId = ‘([a-f0-9-]+)’;
  • Template: $1$
  • Match No. (0 for Random): 1
  • Default Value: null

Configuring variable extraction

Defining load levels using a constant throughput timer

When load testing I find it useful to run a number of tests at different levels of load – for a very specifically defined number of requests per second. To achieve this level of control, the Constant Throughput Timer element is very handy. This timer allows you to specific a target throughput in samples per minute. JMeter will then throttle back requests, if needed, to attempt to achieve this rate. The only thing you may need to do is ensure you have enough threads allowed to meet the desired throughput. Remember that the number of threads can be defined within the Thread Group settings.

Timer settings

Enjoy your load testing!