New testing framework

New testing framework

If you keep an eye on the timeline, you probably saw that the symfony unit tests have been completely reworked lately. This is because we switched from simpletest, which was fine but had side effects when functional tests were executed all at once, to our own testing framework, lime.

Lime is more lighweight than PHPUnit or simpletest and has several advantages. First, it launches test files in a sandbox to avoid strange effects between each test file run (one of the reasons we were unable to fix the old symfony core tests). It also introduces a new sfBrowser, sfTestBrowser and more importantly sfDomCssSelectorBrowser that allow you to write functionnal tests with ease. It is not backward compatible but is a lot more powerful than the old system. Oh, and it holds in a single file, lime.php, without any dependence.

So, you can keep your unit tests written in simpletest but functionnal tests that rely on the sfTestBrowser class have to be upgraded.

Here is a simple usage example of the new sfTestBrowser class for functional tests:

define('SF_ROOT_DIR',    realpath(dirname(__FILE__).'/..'));
define('SF_APP',         $app);
define('SF_ENVIRONMENT', 'test');
define('SF_DEBUG',       true);

require_once(SF_ROOT_DIR.DIRECTORY_SEPARATOR.'apps'.DIRECTORY_SEPARATOR.SF_APP.DIRECTORY_SEPARATOR.'config'.DIRECTORY_SEPARATOR.'config.php');

$b = new sfTestBrowser();
$b->initialize();

// default main page
$b->
   get('/')->
   isStatusCode(200)->
   isRequestParameter('module', 'default')->
   isRequestParameter('action', 'index')->
   checkResponseElement('body', '/congratulations/i') ;

As you can see, the browser now has a fluent interface. The above code is the same as:

$b->get('/');
$b->isStatusCode(200);
$b->isRequestParameter('module', 'default'); 
$b->isRequestParameter('action', 'index'); 
$b->checkResponseElement('body', '/congratulations/i');

The browser object also gives access to the response context for each request after a ->get() or a ->post(). We added shortcuts for testing request parameters or HTTP headers:

$b->isRequestParameter('module', 'action'); 
$b->isResponseHeader('content-type', 'text/html; charset=utf-8');

The new ->checkResponseElement() method is the most powerful of the new methods. It allows to test the response content by CSS selectors. Here are some examples (from sfDomCssSelectorTest unit test file):

$t->is($c->getTexts('h1'), array('Test page'), '->getTexts() takes a CSS selector as its first argument'); 
$t->is($c->getTexts('#footer'), array('footer'), '->getTexts() supports searching html elements by id'); 
$t->is($c->getTexts('.header'), array('header'), '->getTexts() supports searching html elements by class name');
$t->is($c->getTexts('div.header'), array(), '->getTexts() supports searching html elements by class name for a tag name');
$t->is($c->getTexts('ul#mylist ul li'), array('element 3', 'element 4'), '->getTexts() supports searching html elements by several selectors');
$t->is($c->getTexts('ul#list li a[class~="foo1"]'), array('link'), '->getTexts() supports checking attribute word matching'); 
$t->is($c->getTexts('ul#list li a[class^="foo1"]'), array('link'), '->getTexts() supports checking attribute starting with'); 
$t->is($c->getTexts('ul#list li a[class$="foobar1"]'), array('link'), '->getTexts() supports checking attribute ending with'); 
$t->is($c->getTexts('ul#list li a[class*="oba"]'), array('link'), '->getTexts() supports checking attribute with *'); 

To ease the reading of tests results, lime uses coloring on compatible systems.

Until we add documentation for lime, you are invited to have a look at the tests already added to the framework (in the test/ folder) and use them as an example for your own unit and functional tests.

One last word: the current alpha version is not stable enough to be used in production (hence the 'Alpha' attribute). We thank all the devs who send us feedback about remaining bugs, but we warn regular users not to upgrade their application to the 0.8 until we tag it at least beta.