Lime 2 alpha released

I am happy to announce the immediate availability of Lime 2 alpha 1! The second version of symfony's very own testing framework has been under heavy development since early July. Many exciting new features have been added since then, and now you have the opportunity to try them out!

In this blog post, I want to outline the most important new features of Lime 2.

Upgrading a Project

Upgrading a symfony project to use Lime 2 is straight forward. You simply need to replace the file lime.php that comes bundled with symfony with a symbolic link to the lime.php that comes bundled with Lime 2.

TIP Lime 2 is nearly completely backwards compatible! The only thing that is not BC is the configuration of the harness and the coverage class. In Lime 1, this was done through public properties which have now been removed. Instead, you can pass these properties as options to the constructor.

First of all, checkout a copy of Lime 2 from SVN:

> svn co http://svn.symfony-project.com/tools/lime/tags/RELEASE_2_0_0_ALPHA1 lib/vendor/Lime2

Now you can replace symfony's lime.php. The following commands assume that symfony is installed in lib/vendor/symfony. Fix the paths if your project directory structure differs.

> cd lib/vendor/symfony/lib/vendor/lime
> mv lime.php lime.php.1.0
> ln -s ../../../../Lime2/lib/lime.php lime.php

Your done! All your tests will now use Lime 2.

Annotation Support

Often it is necessary to execute some code before every test to prepare the test bed. This code, also called the fixture setup, had to be written manually before every single test in Lime 1. To avoid this code duplication, Lime 2 features annotations to structure and control your test code.

[php]
<?php

require_once dirname(__FILE__).'/../boostrap/unit.php';
LimeAnnotationSupport::enable();
$t = new LimeTest(1);

// @Before

  copy('data/fixtures/test.png', 'web/uploads/test.png');
  $thumbnail = new CukeetThumbnail('web/uploads/test.png');

// @After

  unlink('web/uploads/test.png');
  unset($thumbnail);

// @Test: resize() resizes the thumbnail

  $thumbnail->resize(100, 100);
  $size = getimagesize($thumbnail->getPath());
  $t->is($size, array(100, 100), 'The image has been resized');

// @Test: save() saves the thumbnail under a different name

  $thumbnail->save();
  ...

The most important annotation is @Test. This annotation marks a piece of test code and optionally takes a comment that is printed on the console. The other annotations are @Before, @After, @BeforeAll and @AfterAll. The first two can be used to mark code that is executed before or after every test. The other two are used to mark code to be run once before or after all tests. All code following an annotation belongs to this annotation until the next one is opened.

Parallel Processing

Lime 2 includes the possibility to execute multiple tests simultaneously, taking advantage of modern multi-core processors. This way, the performance of test suite runs can be dramatically improved.

This functionality is not available from the symfony tasks yet. Instead, you need to manually setup a test suite. To do so, add the following code to a script called prove.php in the directory test/bin:

[php]
<?php

include dirname(__FILE__).'/../bootstrap/unit.php';

$h = new LimeTestSuite();
$h->register(sfFinder::type('file')->name('*Test.php')->in(dirname(__FILE__).'/..'));

exit($h->run() ? 0 : 1);

Execute the test suite from a console window:

> php test/bin/prove.php

All your tests should execute as usual. Now add the switch --processes to enable parallel processing:

> php test/bin/prove.php --processes=16

The performance of the test suite should be better now. We'd be happy if you shared your personal performance gain in the comments!

Powerful Mock And Stub Generation

Lime 2 features one of the most powerful yet easy to use mock and stub generators available in PHP. Usually you want to test your classes without testing any other classes that they depend on. This is why these other classes are usually replaced by fake implementations in tests, also called Mock and Stub objects. Because coding these fake implementations takes a lot of time, Lime 2 generates fake implementations for you.

To create a Stub or Mock object call stub() or mock() on your LimeTest object:

[php]
$user = $t->stub('sfUser');

The basic difference between Stubs and Mocks is that Stubs ignore any unexpected method calls by default while Mocks throw exceptions in this case.

To configure a method call, just call the method with the expected parameters. If you don't care about the parameters, pass the method name to any():

[php]
$user->setAttribute('foo');
$user->any('getAttribute');

You can configure method return values, exceptions and forward method calls to callables:

[php]
$user->getAttribute('foo')->returns('bar');
$user->getAttribute('moo')->throws('Exception');

function testGetAttribute($attribute) { ... }
$user->any('getAttribute')->callback('testGetAttribute');

After configuring the expected method calls, you have to call replay(). Only now your object will behave as configured. Optionally you can call verify() after executing the test to check whether all configured methods have been called.

[php]
$user = $t->mock('sfUser');
$user->getAttribute('username')->returns('bernhard');
$user->setAttribute('authorized', true);
$user->replay();

$form = new LoginForm($user);
// internally calls getAttribute() and setAttribute()    
$form->save();

$user->verify();

When you want to test exactly how often a method was called, use either of the count constraints:

[php]
$user->getAttribute('foo')->never();
$user->getAttribute('foo')->once();
$user->getAttribute('foo')->atLeastOnce();
$user->getAttribute('foo')->times(3);
$user->getAttribute('foo')->between(2, 5);

If you want to test single method parameters, use parameter() with any of the test operators (like is(), like() etc.) available in LimeTest:

[php]
$mailer = $t->mock('sfMailer');
$mailer->any('compose')
       ->parameter(2)->is('bernhard.schussek@symfony-project.com')
       ->parameter(4)->like('/Your activation code is ABCXYZ/')
       ->returns($message = $t->stub('Swift_Message'));
$mailer->send($message);

More information about the Mock and Stub generator will be available in the upcoming documentation.

Test Operator Overloading

If you ever tried comparing two Doctrine objects with is(), you have probably seen that the tests almost always fail.

[php]
$user = new User();
$user->fromArray(array('username' => 'bernhard'));
$user->save();

$result = Doctrine::getTable('User')->findOneByUsername('bernhard');

$t->is($result, $user, 'The correct user was returned');

The problem is (from a testing point of view) that Doctrine stores a lot of metadata in the records that differ from record to record, even if both contain the same properties, primary key and relations.

Lime 2 features support for overloading the test operators for specific data types. You can implement your own "tester" class that specifies when the operator should match for a value of this type.

[php]
class myTesterDoctrineRecord extends LimeTesterObject
{
  /**
   * Matches when two Doctrine records have the same primary key,
   * attributes and relations.
   */
  public function is(LimeTester $otherValue) { ... }
}

LimeTester::register('Doctrine_Record', 'myTesterDoctrineRecord');

$t->is($result, $user, 'The correct user was returned');

The supported datatypes are null, integer, boolean, string, double, array, object, resource and any class or interface name of your choice.

What's Next?

Many more features were added to Lime 2. These will be explained in further blog posts and the upcoming documentation. In the next weeks, a CLI tool for executing Lime tests in a developer friendly way will be implemented, which is the last major planned feature before entering beta stage.

Lime 2 is expected to enter beta stage in December or January 2010, depending on the amount of developer feedback on the alpha releases. You are warmly invited to check out the source of Lime 2, play around with it and give feedback on the symfony-users mailing list or in the symfony Trac. Just keep in mind that the code is still alpha, so please don't use it in production.