Playing with symfony's Config Cache

by Kris Wallsmith

One of my personal goals as a symfony developer is to streamline each of my peer's workflow as much as possible on any given project. While I may know our codebase inside and out, that's not a reasonable expectation for everyone on the team. Thankfully, symfony provides mechanisms for isolating and centralizing functionality within a project, making it easy for others to make changes with a very light footprint.

Form Strings

An excellent example of this is the symfony form framework. The form framework is a powerful component of symfony that gives you great control over your forms by moving their rendering and validation into PHP objects. This is a godsend for the application developer, because it means you can encapsulate complex logic in a single form class and extend and reuse it in multiple places.

However, from a template developer's perspective, this abstraction of how a form renders can be troublesome. Take a look at the following form:

The class that configures this form looks something like this:

// lib/form/CommentForm.class.php
class CommentForm extends BaseForm
{
  public function configure()
  {
    $this->setWidget('body', new sfWidgetFormTextarea());
    $this->setValidator('body', new sfValidatorString(array(
      'min_length' => 12,
    )));
  }
}

The form is then rendered in a PHP template like this:

<!-- apps/frontend/modules/main/templates/indexSuccess.php -->
<form action="#" method="post">
  <ul>
    <li>
      <?php echo $form['body']->renderLabel() ?>
      <?php echo $form['body'] ?>
      <?php echo $form['body']->renderError() ?>
    </li>
  </ul>
  <p><button type="submit">Post your comment now</button></p>
</form>

The template developer has quite a bit of control over how this form is rendered. He can change the default labels to be a bit more friendly:

<?php echo $form['body']->renderLabel('Please enter your comment') ?>

He can add a class to the input fields:

<?php echo $form['body']->render(array('class' => 'comment')) ?>

These modifications are intuitive and easy. But what if he needs to modify an error message?

The ->renderError() method does not accept any arguments, so the template developer's only recourse is to open the form's class file, find the code that creates the validator in question, and modify its constructor so the new error messages are associated with the appropriate error codes.

In our example, the template developer would have to make the following change:

// before
$this->setValidator('body', new sfValidatorString(array(
  'min_length' => 12,
)));
 
// after
$this->setValidator('body', new sfValidatorString(array(
  'min_length' => 12,
), array(
  'min_length' => 'You haven't written enough',
)));

Notice a problem? Oops! I used an apostrophe inside a single-quoted string. Of course you or I would never make such a silly mistake, but what's to say a template developer mucking around inside a form class won't?

In all seriousness, can we expect template developers to know their way around the symfony form framework well enough to pinpoint exactly where an error message is defined? Should someone working in the view layer be expected to know the signature for a validator's constructor?

I'm pretty sure we can all agree that the answer to these questions is no. Template developers do a lot of valuable work but it's simply unreasonable to expect someone who isn't writing application code to learn the inner-workings of the symfony form framework.

YAML: A Solution

To simplify the process of editing form strings we are going to add a layer of YAML configuration that enhances each form object as it's passed to the view. This configuration file will look something like this:

# config/forms.yml
CommentForm:
  body:
    label:        Please enter your comment
    attributes:   { class: comment }
    errors:
      min_length: You haven't written enough

This is a lot easier, right? The configuration explains itself, plus the apostrophe issue we encountered earlier is now moot. So let's build it!

Filtering Template Variables

The first challenge is to find a hook in symfony that will allow us to filter every form variable passed to a template through this configuration. To do this, we use the template.filter_parameters event, which is fired from the symfony core just prior to rendering a template or template partial.

// lib/form/sfFormYamlEnhancer.class.php
class sfFormYamlEnhancer
{
  public function connect(sfEventDispatcher $dispatcher)
  {
    $dispatcher->connect('template.filter_parameters',
      array($this, 'filterParameters'));
  }
 
  public function filterParameters(sfEvent $event, $parameters)
  {
    foreach ($parameters as $name => $param)
    {
      if ($param instanceof sfForm && !$param->getOption('is_enhanced'))
      {
        $this->enhance($param);
        $param->setOption('is_enhanced', true);
      }
    }
 
    return $parameters;
  }
 
  public function enhance(sfForm $form)
  {
    // ...
  }
}

This enhancer class needs to be connected from your application configuration:

// apps/frontend/config/frontendConfiguration.class.php
class frontendConfiguration extends sfApplicationConfiguration
{
  public function initialize()
  {
    $enhancer = new sfFormYamlEnhancer($this->getConfigCache());
    $enhancer->connect($this->dispatcher);
  }
}

Now that we're able to isolate form variables just before they're passed to a template or partial we have everything we need to make this work. The final task is to apply what's been configured in the YAML.

Applying the YAML

The easiest way to apply this YAML configuration to each form is to load it into an array and loop through each configuration:

public function enhance(sfForm $form)
{
  $config = sfYaml::load(sfConfig::get('sf_config_dir').'/forms.yml');
 
  foreach ($config as $class => $fieldConfigs)
  {
    if ($form instanceof $class)
    {
      foreach ($fieldConfigs as $fieldName => $fieldConfig)
      {
        if (isset($form[$fieldName]))
        {
          if (isset($fieldConfig['label']))
          {
            $form->getWidget($fieldName)->setLabel($fieldConfig['label']);
          }
 
          if (isset($fieldConfig['attributes']))
          {
            $form->getWidget($fieldName)->setAttributes(array_merge(
              $form->getWidget($fieldName)->getAttributes(),
              $fieldConfig['attributes']
            ));
          }
 
          if (isset($fieldConfig['errors']))
          {
            foreach ($fieldConfig['errors'] as $code => $msg)
            {
              $form->getValidator($fieldName)->setMessage($code, $msg);
            }
          }
        }
      }
    }
  }
}

There are a number of problems with this implementation. First, the YAML file is read from the filesystem and loaded into sfYaml every time a form is enhanced. Reading from the filesystem in this fashion should be avoided. Second, there are multiple levels of nested loops and a number of conditionals that will only slow your application down. The solution for both of these problems lies in symfony's config cache.

The Config Cache

The config cache is composed of a collection of classes that optimize the use of YAML configuration files by automating their translation into PHP code and writing that code to the cache directory for execution. This mechanism will eliminate the overhead necessary to load the contents of our configuration file into sfYaml before applying its values.

Let's implement a config cache for our form enhancer. Instead of loading forms.yml into sfYaml, let's ask the current application's config cache for a pre-processed version.

To do this the sfFormYamlEnhancer class will need access to the current application's config cache, so we'll add that to the constructor.

class sfFormYamlEnhancer
{
  protected
    $configCache = null;
 
  public function __construct(sfConfigCache $configCache)
  {
    $this->configCache = $configCache;
    $this->configCache->registerConfigHandler('config/forms.yml',
      'sfSimpleYamlConfigHandler');
  }
 
  // ...
}

The custom configuration handler could be registered either in the constructor of the form enhancer class as above or in a config_handlers.yml file located in one of your config directory:

# config/config_handlers.yml
config/forms.yml
  class: sfSimpleYamlConfigHandler
  file: %SF_ROOT_DIR%/lib/config/sfSimpleYamlConfigHandler.class.php

The config cache needs to be told what to do when a certain configuration file is requested by the application. For now we've instructed the config cache to use sfSimpleYamlConfigHandler to process forms.yml. This config handler simply parses YAML into an array and caches it as PHP code.

With the config cache in place and a config handler registered for forms.yml we can now call it instead of sfYaml:

public function enhance(sfForm $form)
{
  $config = include $this->configCache->checkConfig('config/forms.yml');
 
  // ...
}

This is much better. Not only have we eliminated the overhead of parsing YAML on all but the first request, we've also switched to using include, which exposes this read to the boons of op-code caching.

Cover me, I'm goin' in!

Let's write some tests before going any further. We can start with this basic script:

// test/unit/form/sfFormYamlEnhancerTest.php
include dirname(__FILE__).'/../../bootstrap/unit.php';
 
$t = new lime_test(3);
 
$configuration = $configuration->getApplicationConfiguration(
  'frontend', 'test', true, null, $configuration->getEventDispatcher());
sfToolkit::clearDirectory(sfConfig::get('sf_app_cache_dir'));
 
$enhancer = new sfFormYamlEnhancer($configuration->getConfigCache());
 
// ->enhance()
$t->diag('->enhance()');
 
$form = new CommentForm();
$form->bind(array('body' => '+1'));
 
$enhancer->enhance($form);
 
$t->like($form['body']->renderLabel(), '/Please enter your comment/',
  '->enhance() enhances labels');
$t->like($form['body']->render(), '/class="comment"/',
  '->enhance() enhances widgets');
$t->like($form['body']->renderError(), '/You haven\'t written enough/',
  '->enhance() enhances error messages');

Running this test against the current sfFormYamlEnhancer verifies that it is working correctly:

Now we can go about refactoring with the confidence that our tests will raise a stink if we break anything.

Custom Config Handlers

In the enhancer code above, every form variable passed to a template will loop through every form class configured in forms.yml. This gets the job done, but if you pass multiple form objects to a template, or have a long list of forms configured in the YAML, you may begin to see an impact on performance. This is a good opportunity to write a custom config handler that optimizes this process.

Most of the magic of config handlers happens behind the scenes. The config cache takes care of the caching logic before it runs any particular config handler so we can just focus on generating the code necessary to apply the YAML configuration.

Each config handler must implement the following two methods:

• static public function getConfiguration(array $configFiles)
• public function execute($configFiles)

The first method, ::getConfiguration(), is passed an array of file paths, parses them and merges their contents into a single value. In the sfSimpleYamlConfigHandler class we used above, this method includes only one line:

static public function getConfiguration(array $configFiles)
{
  return self::parseYamls($configFiles);
}

The sfSimpleYamlConfigHandler class extends the abstract sfYamlConfigHandler which includes a number of helper methods for processing YAML configuration files:

• ::parseYamls($configFiles)
• ::parseYaml($configFile)
• ::flattenConfiguration($config)
• ::flattenConfigurationWithEnvironment($config)

The first two methods implement symfony's configuration cascade. The second two implement environment-awareness.

The ::getConfiguration() method in our config handler will need a custom method for merging the configuration based on class inheritance. Create an ::applyInheritance() method that encapsulates this logic:

// lib/config/sfFormYamlEnhancementsConfigHandler.class.php
class sfFormYamlEnhancementsConfigHandler extends sfYamlConfigHandler
{
  public function execute($configFiles)
  {
    $config = self::getConfiguration($configFiles);
 
    // compile data
    $retval = "<?php\n".
              "// auto-generated by %s\n".
              "// date: %s\nreturn %s;\n";
    $retval = sprintf($retval, __CLASS__, date('Y/m/d H:i:s'),
      var_export($config, true));
 
    return $retval;
  }
 
  static public function getConfiguration(array $configFiles)
  {
    return self::applyInheritance(self::parseYamls($configFiles));
  }
 
  static public function applyInheritance($config)
  {
    $classes = array_keys($config);
 
    $merged = array();
    foreach ($classes as $class)
    {
      if (class_exists($class))
      {
        $merged[$class] = $config[$class];
        foreach (array_intersect(class_parents($class), $classes) as $parent)
        {
          $merged[$class] = sfToolkit::arrayDeepMerge(
            $config[$parent],
            $merged[$class]
          );
        }
      }
    }
 
    return $merged;
  }
}

We now have an array whose values have been merged per class inheritance. This eliminates the need to filter the entire configuration through instanceof for each form object. What's more, this merge is done in the config handler so it will only happen once and then be cached.

Now we can apply this merged array to a form object with a simple bit of search logic:

class sfFormYamlEnhancer
{
  protected
    $configCache = null;
 
  public function __construct(sfConfigCache $configCache)
  {
    $this->configCache = $configCache;
    $this->configCache->registerConfigHandler('config/forms.yml',
      'sfFormYamlEnhancementsConfigHandler');
  }
 
  // ...
 
  public function enhance(sfForm $form)
  {
    $config = include $this->configCache->checkConfig('config/forms.yml');
 
    $class = get_class($form);
    if (isset($config[$class]))
    {
      $fieldConfigs = $config[$class];
    }
    else if ($overlap = array_intersect(class_parents($class),
      array_keys($config)))
    {
      $fieldConfigs = $config[current($overlap)];
    }
    else
    {
      return;
    }
 
    foreach ($fieldConfigs as $fieldName => $fieldConfig)
    {
      // ...
    }
  }
}

Before we run the test script again, let's add an assertion for the new class inheritance logic.

# config/forms.yml
 
# ...
 
BaseForm:
  body:
    errors:
      min_length: A base min_length message
      required:   A base required message

We can verify that the new required message is being applied in the test script, and confirm that child forms will receive their parents' enhancements, even if there are none configured for the child class.

$t = new lime_test(5);
 
// ...
 
$form = new CommentForm();
$form->bind();
$enhancer->enhance($form);
$t->like($form['body']->renderError(), '/A base required message/',
  '->enhance() considers inheritance');
 
class SpecialCommentForm extends CommentForm { }
$form = new SpecialCommentForm();
$form->bind();
$enhancer->enhance($form);
$t->like($form['body']->renderLabel(), '/Please enter your comment/',
  '->enhance() applies parent config');

Run this updated test script to verify the form enhancer is working as expected.

Getting Fancy with Embedded Forms

There is an important feature of the symfony form framework we haven't considered yet: embedded forms. If an instance of CommentForm is embedded in another form, the enhancements we've made in forms.yml will not be applied. This is easy enough to demonstrate in our test script:

$t = new lime_test(6);
 
// ...
 
$form = new BaseForm();
$form->embedForm('comment', new CommentForm());
$form->bind();
$enhancer->enhance($form);
$t->like($form['comment']['body']->renderLabel(),
  '/Please enter your comment/',
  '->enhance() enhances embedded forms');

This new assertion demonstrates that embedded forms are not being enhanced:

Fixing this test will involve a more advanced config handler. We need to be able to apply the enhancements configured in forms.yml in a modular way to account for embedded forms, so we are going to generate a tailored enhancer method for each configured form class. These methods will be generated by our custom config handler in a new "worker" class.

class sfFormYamlEnhancementsConfigHandler extends sfYamlConfigHandler
{
  // ...
 
  protected function getEnhancerCode($fields)
  {
    $code = array();
    foreach ($fields as $field => $config)
    {
      $code[] = sprintf('if (isset($fields[%s]))', var_export($field, true));
      $code[] = '{';
 
      if (isset($config['label']))
      {
        $code[] = sprintf('  $fields[%s]->getWidget()->setLabel(%s);',
          var_export($config['label'], true));
      }
 
      if (isset($config['attributes']))
      {
        $code[] = '  $fields[%s]->getWidget()->setAttributes(array_merge(';
        $code[] = '    $fields[%s]->getWidget()->getAttributes(),';
        $code[] = '    '.var_export($config['attributes'], true);
        $code[] = '  ));';
      }
 
      if (isset($config['errors']))
      {
        $code[] = sprintf('  if ($error = $fields[%s]->getError())',
          var_export($field, true));
        $code[] = '  {';
        $code[] = '    $error->getValidator()->setMessages(array_merge(';
        $code[] = '      $error->getValidator()->getMessages(),';
        $code[] = '      '.var_export($config['errors'], true);
        $code[] = '    ));';
        $code[] = '  }';
      }
 
      $code[] = '}';
    }
 
    return implode(PHP_EOL.'    ', $code);
  }
}

Notice how the config array is checked for certain keys when the code is generated, rather than at runtime. This will provide a small performance boost.

This generated code is placed inside a class definition, which is then saved to the cache directory.

class sfFormYamlEnhancementsConfigHandler extends sfYamlConfigHandler
{
  public function execute($configFiles)
  {
    $forms = self::getConfiguration($configFiles);
 
    $code = array();
    $code[] = '<?php';
    $code[] = '// auto-generated by '.__CLASS__;
    $code[] = '// date: '.date('Y/m/d H:is');
    $code[] = 'class sfFormYamlEnhancementsWorker';
    $code[] = '{';
    $code[] = '  static public $enhancable = '.var_export(array_keys($forms), true).';';
 
    foreach ($forms as $class => $fields)
    {
      $code[] = '  static public function enhance'.$class.'(sfFormFieldSchema $fields)';
      $code[] = '  {';
      $code[] = '    '.$this->getEnhancerCode($fields);
      $code[] = '  }';
    }
 
    $code[] = '}';
 
    return implode(PHP_EOL, $code);
  }
 
  // ...
}

The sfFormYamlEnhancer class will now defer to the generated worker class to handle manipulation of form objects, but must now account for recursion through embedded forms. To do this we must process the form's field schema (which can be iterated through recursively) and the form object (which includes the embedded forms) in parallel.

class sfFormYamlEnhancer
{
  // ...
 
  public function enhance(sfForm $form)
  {
    require_once $this->configCache->checkConfig('config/forms.yml');
    $this->doEnhance($form->getFormFieldSchema(), $form);
  }
 
  protected function doEnhance(sfFormFieldSchema $fieldSchema, sfForm $form)
  {
    if ($enhancer = $this->getEnhancer(get_class($form)))
    {
      call_user_func($enhancer, $fieldSchema);
    }
 
    foreach ($form->getEmbeddedForms() as $name => $form)
    {
      if (isset($fieldSchema[$name]))
      {
        $this->doEnhance($fieldSchema[$name], $form);
      }
    }
  }
 
  public function getEnhancer($class)
  {
    if (in_array($class, sfFormYamlEnhancementsWorker::$enhancable))
    {
      return array('sfFormYamlEnhancementsWorker', 'enhance'.$class);
    }
    else if ($overlap = array_intersect(class_parents($class),
      sfFormYamlEnhancementsWorker::$enhancable))
    {
      return array('sfFormYamlEnhancementsWorker', 'enhance'.current($overlap));
    }
  }
}

With support for embedded forms in place, our tests should now be passing. Run the script to find out:

How'd we do?

Let's run some benchmarks just to be sure we haven't wasted our time. To make the results interesting, add a few more form classes to forms.yml using a PHP loop.

# <?php for ($i = 0; $i < 100; $i++): ?> #
Form<?php echo $i ?>: ~
# <?php endfor; ?> #

Create all these classes by running the following snippet of code:

mkdir($dir = sfConfig::get('sf_lib_dir').'/form/test_fixtures');
for ($i = 0; $i < 100; $i++)
{
  file_put_contents($dir.'/Form'.$i.'.class.php',
    '<?php class Form'.$i.' extends BaseForm { }');
}

Now we're ready to run some benchmarks. For the results below, I've ran the following Apache command on my MacBook multiple times until I got a standard deviation of less than 2ms.

$ ab -t 60 -n 20 http://localhost/config_cache/web/index.php

Start with a baseline benchmark for running the application without the enhancer connected at all. Comment out sfFormYamlEnhancer in frontendConfiguration and run the benchmark:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    62   63   1.5     63      69
Waiting:       62   63   1.5     63      69
Total:         62   63   1.5     63      69

Next, paste the first version of sfFormYamlEnhancer::enhance() that called sfYaml directly into the class and run the benchmark:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    87   88   1.6     88      93
Waiting:       87   88   1.6     88      93
Total:         87   88   1.7     88      94

You can see we've added an average of 25ms to each request, an increase of almost 40%. Next, undo the change you just made to ->enhance() so our custom config handler is restored and run the benchmark again:

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       0
Processing:    62   63   1.6     63      70
Waiting:       62   63   1.6     63      70
Total:         62   64   1.6     63      70

As you can see, we've reduced processing time back to the baseline by creating a custom config handler.

Just For Fun: Bundling a Plugin

Now that we have this great system in place for enhancing form objects with a simple YAML configuration file, why not bundle it up as a plugin and share it with the community. This may sound intimidating to those who haven't published a plugin in the past; hopefully we can dispell some of that fear now.

This plugin will have the following file structure:

sfFormYamlEnhancementsPlugin/
  config/
    sfFormYamlEnhancementsPluginConfiguration.class.php
  lib/
    config/
      sfFormYamlEnhancementsConfigHandler.class.php
    form/
      sfFormYamlEnhancer.class.php
  test/
    unit/
      form/
        sfFormYamlEnhancerTest.php

We need to make a few modifications to ease the plugin installation process. Creation and connection of the enhancer object can be encapsulated in the plugin configuration class:

class sfFormYamlEnhancementsPluginConfiguration extends sfPluginConfiguration
{
  public function initialize()
  {
    if ($this->configuration instanceof sfApplicationConfiguration)
    {
      $enhancer = new sfFormYamlEnhancer($this->configuration->getConfigCache());
      $enhancer->connect($this->dispatcher);
    }
  }
}

The test script will need to be updated to reference the project's bootstrap script:

include dirname(__FILE__).'/../../../../../test/bootstrap/unit.php';
 
// ...

Finally, enable the plugin in ProjectConfiguration:

class ProjectConfiguration extends sfProjectConfiguration
{
  public function setup()
  {
    $this->enablePlugins('sfFormYamlEnhancementsPlugin');
  }
}

If you want to run tests from the plugin, connect them in ProjectConfiguration now:

class ProjectConfiguration extends sfProjectConfiguration
{
  // ...
 
  public function setupPlugins()
  {
    $this->pluginConfigurations['sfFormYamlEnhancementsPlugin']->connectTests();
  }
}

Now the tests from the plugin will run when you call any of the test:* tasks.

All of our classes are now located in the new plugin's directory, but there is one problem: the test script relies on files that are still located in the project. This means that anyone else who may want to run these tests would not be able to unless they have the same files in their project.

To fix this we'll need to isolate the code in the enhancer class that calls the config cache so we can overload it in our test script and instead use a forms.yml fixture.

class sfFormYamlEnhancer
{
  // ...
 
  public function enhance(sfForm $form)
  {
    $this->loadWorker();
    $this->doEnhance($form->getFormFieldSchema(), $form);
  }
 
  public function loadWorker()
  {
    require_once $this->configCache->checkConfig('config/forms.yml');
  }
 
  // ...
}

We can then overload the ->loadWorker() method in our test script to call the custom config handler directly. The CommentForm class should also be moved to the test script and the forms.yml file moved to the plugin's test/fixtures directory.

include dirname(__FILE__).'/../../../../../test/bootstrap/unit.php';
 
$t = new lime_test(6);
 
class sfFormYamlEnhancerTest extends sfFormYamlEnhancer
{
  public function loadWorker()
  {
    if (!class_exists('sfFormYamlEnhancementsWorker', false))
    {
      $configHandler = new sfFormYamlEnhancementsConfigHandler();
      $code = $configHandler->execute(array(dirname(__FILE__).'/../../fixtures/forms.yml'));
 
      $file = tempnam(sys_get_temp_dir(), 'sfFormYamlEnhancementsWorker');
      file_put_contents($file, $code);
 
      require $file;
    }
  }
}
 
class CommentForm extends BaseForm
{
  public function configure()
  {
    $this->setWidget('body', new sfWidgetFormTextarea());
    $this->setValidator('body', new sfValidatorString(array('min_length' => 12)));
  }
}
 
$configuration = $configuration->getApplicationConfiguration(
  'frontend', 'test', true, null, $configuration->getEventDispatcher());
 
$enhancer = new sfFormYamlEnhancerTest($configuration->getConfigCache());
 
// ...

Finally, packaging the plugin is easy with sfTaskExtraPlugin installed. Just run the plugin:package task and a package will be created after a few interactive prompts.

$ php symfony plugin:package sfFormYamlEnhancementsPlugin

Final Thoughts

As you can see by the benchmarks done here, the symfony config cache makes it possible to utilize the simplicity of YAML configuration files with virtually no impact on performance.