About testing stuff...

There has been some discussion on Twitter about the quality vs. quantity of symfony plugins out there. This is a good opportunity to touch on one of the new features introduced in symfony 1.3 that allows you to hook tests from plugins into the symfony test:* commands with just a few lines of code.

The sfPluginConfiguration class, introduced in symfony 1.2, now includes a connectTests() method. This method, as you can tell by its name, connects tests from the plugin to the current project. You can call this method from your ProjectConfiguration class:

[php]
// config/ProjectConfiguration.class.php
class ProjectConfiguration extends sfProjectConfiguration
{
  public function setup()
  {
    $this->enablePlugins('myPlugin');
  }

  public function setupPlugins()
  {
    $this->pluginConfigurations['myPlugin']->connectTests();
  }
}

With this code in place, tests from myPlugin will be included when any of symfony's test:* tasks are run:

$ php symfony test:all
$ php symfony test:unit
$ php symfony test:unit myPluginClass
$ php symfony test:functional frontend
$ php symfony test:functional frontend myPluginModuleActions

You can add your own logic to this connecting code. For example, if you use prefixed plugins as a means to organize project code, you can run a quick loop to connect all of your project-plugins' tests:

[php]
public function setupPlugins()
{
  foreach ($this->plugins as $plugin)
  {
    // check for your project's prefix
    if (0 === strpos($plugin, 'my'))
    {
      $this->pluginConfigurations[$plugin]->connectTests();
    }
  }
}

Something Extra

If your project uses sfTaskExtraPlugin, a plugin maintained by the core team, you have access to the test:plugin command. This is the best way to run a particular plugin's entire test suite.

$ php symfony test:plugin myPlugin

NOTE While the core test:* commands require you to call sfPluginConfiguration::connectTests() from your project configuration in order to include tests from a plugin, the test:plugin command does not. It is still required that the plugin be enabled.

The task-extra plugin also includes a generate:test task, which makes it easy to get a new unit test started quickly:

$ php symfony generate:test myPluginClass --editor-cmd=mate

This command will create a stub unit test in a myPluginClassTest.php file nested in the plugin's test/unit/ directory in such a way to replicate the directory structure of lib/. Having the test directory organized as a mirror of the lib directory makes it possible to run the test:coverage command if you have Xdebug installed:

$ php symfony test:coverage test/unit/util lib/util --detailed

Step-by-step

Let's assume you're considering sfFormExtraPlugin for your project. One way to evaluate this plugin is by evaluating the quality of its test suite. This can be done in three easy steps:

  1. Install the plugin:

    $ php symfony plugin:install sfFormExtraPlugin
    
  2. Run the plugin's test suite:

    $ php symfony test:plugin sfFormExtraPlugin
    
  3. Check the plugin's test coverage:

    $ php symfony test:coverage plugins/sfFormExtraPlugin/test \
      plugins/sfFormExtraPlugin/lib
    

Go forth and test!

Symfony has always advocated for writing automated tests and provided the tools necessary to do so in your project, but support for testing plugins has been limited. Hopefully, with these enhancements introduced in symfony 1.3 and sfTaskExtraPlugin, more plugin developers will include test suites with their plugins, and more users will run them.

Comments

Didn't know about it, I copied the tests into application test folders all the time :D
That makes me think it would be awesome to have a dedicated sismo instance to run hosted plugins test suites ; kinda continuous integration for the masses, would at last provide some visibility over the quality of contributes plugins...

Okay, that's huge task and would need tons of machines to run all those tests periodically, but let me dream a minute ;)
I agree with NiKo that symfony plugins testing on symfony servers would be nice - but it wouldn't be that difficult, imho. There's no need to run tests periodically - only when a new release of plugin is submitted/svn-committed (or when new version of symfony is released).

As for me, there's one big issue with plugins in general - there are LOTS of plugins, some of them are really good, but sometimes it's really difficult to find them. It's like finding money on the ground - you're either lucky or not. That's because very few people mark that they really do use symfony plugins. Top plugins have less than 500 users. Can you believe it? I don't :) Maybe it's 100 times more than the stats show now?

The result is that only top 10 (or top 30) plugins out of almost 1k are well-known. There should be more features to tell if a plugin is good value. Maybe 'download count' (using either downloading from www.symfony-project.org .tar.gz or CLI symfony plugin:install)? Those numbers would be a lot more real. This would be automatical, not so much code to write. We must realize that symfony community plugins are becoming a great powerful tool...

Another option to rate plugins is NiKo's idea. What do you think?
dfdf
Tomasz, this is one of the issues http://symplist.net is trying to fix. It is still in alpha, but needs help from the community to come up with solid ratings, plugin feedback, and site feedback in general!

Great post, Kris. Very useful stuff.
hmm.. I just wrote two modules in myPlugin 'news' and 'newsAdmin'. The first being enabled in the frontend application, the latter in the backend.

running:
$ symfony test:functional
will execute both no matter what application you try. Which is wrong, because for frontend and backend one test will fail.

I was able to bypass this by running:
$ symfony test:functional frontend frontend/*
and
$ symfony test:functional backend backend/*

This way the tests get only executed for the appropriate application (depending on folder organisation).

However now the --xml=test-results.xml option does not produce an XML.

Comments are closed.

To ensure that comments stay relevant, they are closed for old posts.