New in Symfony 2.8: Clock mocking and time sensitive tests

Contributed by
Nicolas Grekas
in #16194.

Transient tests are those which fail randomly depending on spurious and external circumstances, such as the underlying system load. These tests are very risky because they make your test suite unreliable.

Tests that deal with time-related functions are one of the most common transient tests. Consider for example the following test:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
use Symfony\Component\Stopwatch\Stopwatch;

class MyTest extends \PHPUnit_Framework_TestCase
{
    public function testSomething()
    {
        $stopwatch = new Stopwatch();

        $stopwatch->start('event');
        sleep(10);
        $duration = $stopwatch->stop()->getDuration();

        $this->assertEquals(10, $duration);
    }
}

This code is so simple that it seems impossible to fail. However, depending on the load of the server, the $duration could be for example 10.00000023 and the test would fail for no apparent reason.

This kind of errors happen frequently when using public continuous integration services like Travis CI. We even have a long-running issue to hunt all these transient tests.

Clock mocking

In order to solve all the time-related test errors, the PHPUnit bridge now includes a ClockMock class which can be used in your PHPUnit tests. This class replaces the PHP's built-in time(), microtime(), sleep() and usleep() functions by its own implementations.

This means that you don't need to make a single change in your original code, except when using new DateTime(), which must be replaced by DateTime::createFromFormat('U', time()) to use the mocked time() function.

The clock mocking is enabled on demand for the tests which need it. The recommended way to enable it is to add a special @group annotation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
/**
 * @group time-sensitive
 */
class MyTest extends \PHPUnit_Framework_TestCase
{
    public function testSomething()
    {
        $stopwatch = new Stopwatch();

        $stopwatch->start('event');
        sleep(10);
        $duration = $stopwatch->stop()->getDuration();

        $this->assertEquals(10, $duration);
    }
}

And that's all! This test will never fail again because of getting a wrong time-related result. The sleep(10) call will make the clock advance 10 exact seconds and your test will always pass.

An added bonus of using the ClockMock class is that time passes instantly. Using PHP's sleep(10) will make your test wait for 10 actual seconds (more or less). In contrast, the ClockMock class advances the internal clock the given number of seconds without actually waiting that time, so your test will execute 10 seconds faster.

Alternative Ways to Enable the Clock Mocking

The @group time-sensitive works "by convention" and assumes that the namespace of the tested class can be obtained just by removing the \Tests\ part from the test namespace.

If this convention doesn't work for your application, you can also configure the mocked namespaces in the phpunit.xml file, as done for example in the HttpKernel component:

1
2
3
4
5
6
7
8
9
<listeners>
    <listener class="Symfony\Bridge\PhpUnit\SymfonyTestsListener">
        <arguments>
            <array>
                <element><string>Symfony\Component\HttpFoundation</string></element>
            </array>
        </arguments>
    </listener>
</listeners>

Lastly, you can also enable clock mocking explicitly. Just call the \Symfony\Bridge\PhpUnit\ClockMock::register() method from setupBeforeClass() and pass the FQCN from which the convention explained before should be applied. Then, pass a boolean argument to ClockMock::withClockMock() method to enable/disable the clock mocking.

Comments

Awesome. That would be great for the integration tests or php-cache.com
Note that the example requires "use Symfony\Component\Stopwatch\Stopwatch;", but even with this, it doesn't work: it returns "Missing argument 1 for Symfony\Component\Stopwatch\Stopwatch::start()". According to the source code, the first argument "$name" is required.
Here is a working test :

$stopwatch->start('a');
sleep(1);
$duration = $stopwatch->stop('a')->getDuration();

$this->assertEquals(1000, $duration);"

I used `@group time-sensitive` on the class and the function and `sleep()` really took one second.
Cool !
@Alexis An implicit requirement for the feature to work is to enable the phpunit-bridge, see https://github.com/symfony/phpunit-bridge
@Nikolas Once I added the configuration in phpunit.xml.dist, as suggested in this post, it worked. So I think the problem come from the test class name.

In the official documentation (http://symfony.com/doc/current/book/testing.html#functional-tests), tests classes names end with "Test". Is it compatible with the fact that ”the namespace of the tested class can be obtained just by removing the \Tests\ part from the test namespace.”?

Can someone please update the post? It looks like there are errors in the example: $stopwatch->start() requires one argument: https://github.com/symfony/symfony/blob/c0e4495b66751984125122ab4093b70521102c93/src/Symfony/Component/Stopwatch/Stopwatch.php#L87-L98 And $stopwatch->stop() doesn't return an integer. Thanks.
@Alexis, I've updated the code examples of the article.
@Javier Thanks! I'm sorry to bother you but "stop()" should be "stop('event')".
@Alexis Yes, it is compatible with any class name because to mock native functions, it only requires to know the namespace of the class which use those native functions. Here is how it was done in PHPUnitBridge: https://github.com/symfony/phpunit-bridge/blob/master/ClockMock.php#L72

Comments are closed.

To ensure that comments stay relevant, they are closed for old posts.