Day 22: The Cache

Today, we will talk about caching. The symfony framework has many built-in cache strategies. For instance, the YAML configuration files are first converted to PHP and then cached on the filesystem. We have also seen that the modules generated by the admin generator are cached for better performance.

But today, we will talk about another cache: the HTML cache. To improve your website performance, you can cache whole HTML pages or just parts of them.

Creating a new Environment

By default, the template cache feature of symfony is enabled in the settings.yml configuration file for the prod environment, but not for the test and dev ones:

prod:
  .settings:
    cache: on
 
dev:
  .settings:
    cache: off
 
test:
  .settings:
    cache: off

As we need to test the cache feature before going to production, we can activate the cache for the dev environment or create a new environment. Recall that an environment is defined by its name (a string), an associated front controller, and optionally a set of specific configuration values.

To play with the cache system on Jobeet, we will create a cache environment, similar to the prod environment, but with the log and debug information available in the dev environment.

Create the front controller associated with the new cache environment by copying the dev front controller web/frontend_dev.php to web/frontend_cache.php:

// web/frontend_cache.php
if (!in_array(@$_SERVER['REMOTE_ADDR'], array('127.0.0.1', '::1')))
{
  die('You are not allowed to access this file. Check '.basename(__FILE__).' for more information.');
}
 
require_once(dirname(__FILE__).'/../config/ProjectConfiguration.class.php');
 
$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'cache', true);
sfContext::createInstance($configuration)->dispatch();

That's all there is to it. The new cache environment is now useable. The only difference is the second argument of the getApplicationConfiguration() method which is the environment name, cache.

You can test the cache environment in your browser by calling its front controller:

http://jobeet.localhost/frontend_cache.php/

For now, the cache environment inherits from the default configuration. Edit the settings.yml configuration file to add the cache environment specific configuration:

# apps/frontend/config/settings.yml
cache:
  .settings:
    error_reporting: <?php echo (E_ALL | E_STRICT)."\n" ?>
    web_debug:       on
    cache:           on
    etag:            off

In these settings, the symfony template cache feature has been activated with the cache setting and the web debug toolbar has been enabled with the web_debug setting.

As we also want to log SQL statements, we need to change the database configuration. Edit databases.yml and add the following configuration at the beginning of the file:

# config/databases.yml
cache:
  propel:
    class: sfPropelDatabase
    param:
      classname: DebugPDO

As the default configuration caches all settings in the cache, you need to clear it before being able to see the changes in your browser:

$ php symfony cc

Now, if you refresh your browser, the web debug toolbar should be present in the top right corner of the page, as it is the case for the dev environment.

Cache Configuration

The symfony template cache can be configured with the cache.yml configuration file. The default configuration for the application is to be found in apps/frontend/config/cache.yml:

default:
  enabled:     off
  with_layout: false
  lifetime:    86400

By default, as all pages can contain dynamic information, the cache is globally disabled (enabled: off). We don't need to change this setting, because we will enable the cache on a page by page basis.

The lifetime setting defines the server side life time of the cache in seconds (86400 seconds equals one day).

Page Cache

As the Jobeet homepage will probably be the most visited page of the website, instead of requesting the data from the database each time a user accesses it, it can be cached.

Create a cache.yml file for the sfJobeetJob module:

# plugins/sfJobeetPlugin/modules/sfJobeetJob/config/cache.yml
index:
  enabled:     on
  with_layout: true

If you refresh your browser, you will see that symfony has decorated the page with a box indicating that the content has been cached:

The box gives some precious information about the cache key for debugging, like the lifetime of the cache, and the age of it.

If you refresh the page again, the color of the box changed from green to yellow, indicating that the page has been retrieved from the cache:

Also notice that no database request has been made in the second case, as shown in the web debug toolbar.

When a page is cacheable, and if the cache does not exist yet, symfony stores the response object in the cache at the end of the request. For all other future requests, symfony will send the cached response without calling the controller:

This has a great impact on performance as you can measure for yourself by using tools like JMeter.

The job creation page can also be cached:

# plugins/sfJobeetPlugin/modules/sfJobeetJob/config/cache.yml
new:
  enabled:     on
 
index:
  enabled:     on
 
all:
  with_layout: true

As the two pages can be cached with the layout, we have created an all section that defines the default configuration for the all sfJobeetJob module actions.

Clearing the Cache

If you want to clear the page cache, you can use the cache:clear task:

$ php symfony cc

The cache:clear task clears all the symfony caches stored under the main cache/ directory. It also takes options to selectively clear some parts of the cache. To only clear the template cache for the cache environment, use the --type and --env options:

$ php symfony cc --type=template --env=cache

Instead of clearing the cache each time you make a change, you can also disable the cache by adding any query string to the URL, or by using the "Ignore cache" button from the web debug toolbar:

Action Cache

Sometimes, you cannot cache the whole page in the cache, but the action template itself can be cached. Put another way, you can cache everything but the layout.

For the Jobeet application, we cannot cache the whole page because of the "history job" bar.

Change the configuration for the job module cache accordingly:

# plugins/sfJobeetPlugin/modules/sfJobeetJob/config/cache.yml
new:
  enabled:     on
 
index:
  enabled:     on
 
all:
  with_layout: false

By changing the with_layout setting to false, you have disabled layout caching.

Clear the cache:

$ php symfony cc

Refresh your browser to see the difference:

Even if the flow of the request is quite similar in the simplified diagram, caching without the layout is much more resource intensive.

Partial and Component Cache

For highly dynamic websites, it is sometimes even impossible to cache the whole action template. For those cases, you need a way to configure the cache at the finer-grained level. Thankfully, partials and components can also be cached.

Let's cache the language component by creating a cache.yml file for the sfJobeetLanguage module:

# plugins/sfJobeetPlugin/modules/sfJobeetLanguage/config/cache.yml
_language:
  enabled: on

Configuring the cache for a partial or a component is as simple as adding an entry with its name. The with_layout option is not taken into account for this type of cache as it does not make any sense:

_sidebar:
  enabled:    on
  contextual: true

Forms in Cache

Storing the job creation page in the cache is problematic as it contains a form. To better understand the problem, go to the "Post a Job" page in your browser to seed the cache. Then, clear your session cookie, and try to submit a job. You must see an error message alerting you of a "CSRF attack":

Why? As we have configured a CSRF secret when we created the frontend application, symfony embeds a CSRF token in all forms. To protect you against CSRF attacks, this token is unique for a given user and for a given form.

The first time the page is displayed, the generated HTML form is stored in the cache with the current user token. If another user comes afterwards, the page from the cache will be displayed with the first user CSRF token. When submitting the form, the tokens do not match, and an error is thrown.

How can we fix the problem as it seems legitimate to store the form in the cache? The job creation form does not depend on the user, and it does not change anything for the current user. In such a case, no CSRF protection is needed, and we can remove the CSRF token altogether:

// plugins/sfJobeetPlugin/lib/form/JobeetJobForm.class.php
class JobeetJobForm extends BaseJobeetJobForm
{
  public function __construct(BaseObject $object = null, $options = array(), $CSRFSecret = null)
  {
    parent::__construct($object, $options, false);
  }
 
  // ...
}

After doing this change, clear the cache and re-try the same scenario as above to prove it works as expected now.

The same configuration must be applied to the language form as it is contained in the layout and will be stored in the cache. As the default sfLanguageForm is used, instead of creating a new class, just to remove the CSRF token, let's do it from the action and component of the sfJobeetLanguage module:

// plugins/sfJobeetPlugin/modules/sfJobeetLanguage/actions/components.class.php
class sfJobeetLanguageComponents extends sfComponents
{
  public function executeLanguage(sfWebRequest $request)
  {
    $this->form = new sfFormLanguage($this->getUser(), array('languages' => array('en', 'fr')));
    unset($this->form[$this->form->getCSRFFieldName()]);
  }
}
 
// plugins/sfJobeetPlugin/modules/sfJobeetLanguage/actions/actions.class.php
class sfJobeetLanguageActions extends sfActions
{
  public function executeChangeLanguage(sfWebRequest $request)
  {
    $form = new sfFormLanguage($this->getUser(), array('languages' => array('en', 'fr')));
    unset($form[$form->getCSRFFieldName()]);
 
    // ...
  }
}

The getCSRFFieldName() returns the name of the field that contains the CSRF token. By unsetting this field, the widget and the associated validator are removed.

Removing the Cache

Each time a user posts and activates a job, the homepage must be refreshed to list the new job.

As we don't need the job to appear in real-time on the homepage, the best strategy is to lower the cache life time to something acceptable:

# plugins/sfJobeetPlugin/modules/sfJobeetJob/config/cache.yml
index:
  enabled:  on
  lifetime: 600

Instead of the default configuration of one day, the cache for the homepage will be automatically removed every ten minutes.

But if you want to update the homepage as soon as a user activates a new job, edit the executePublish() method of the sfJobeetJob module to add manual cache cleaning:

// plugins/sfJobeetPlugin/modules/sfJobeetJob/actions/actions.class.php
public function executePublish(sfWebRequest $request)
{
  $request->checkCSRFProtection();
 
  $job = $this->getRoute()->getObject();
  $job->publish();
 
  if ($cache = $this->getContext()->getViewCacheManager())
  {
    $cache->remove('sfJobeetJob/index?sf_culture=*');
    $cache->remove('sfJobeetCategory/show?id='.$job->getJobeetCategory()->getId());
  }
 
  $this->getUser()->setFlash('notice', sprintf('Your job is now online for %s days.', sfConfig::get('app_active_days')));
 
  $this->redirect($this->generateUrl('job_show_user', $job));
}

The cache is managed by the sfViewCacheManager class. The remove() method removes the cache associated with an internal URI. To remove cache for all possible parameters of a variable, use the * as the value. The sf_culture=* we have used in the code above means that symfony will remove the cache for the English and the French homepage.

As the cache manager is null when the cache is disabled, we have wrapped the cache removing in an if block.

$user = sfContext::getInstance()->getUser();

sfContext::getInstance()->set('job', $job);
$job = sfContext::getInstance()->get('job');

Testing the Cache

Before starting, we need to change the configuration for the test environment to enable the cache layer:

# apps/frontend/config/settings.yml
test:
  .settings:
    error_reporting: <?php echo ((E_ALL | E_STRICT) ^ E_NOTICE)."\n" ?>
    cache:           on
    web_debug:       off
    etag:            off

Let's test the job creation page:

// test/functional/frontend/jobActionsTest.php
$browser->
  info('  7 - Job creation page')->
 
  get('/fr/')->
  with('view_cache')->isCached(true, false)->
 
  createJob(array('category_id' => $browser->getProgrammingCategory()->getId()), true)->
 
  get('/fr/')->
  with('view_cache')->isCached(true, false)->
  with('response')->checkElement('.category_programming .more_jobs', '/23/')
;

The view_cache tester is used to test the cache. The isCached() method takes two booleans:

• Whether the page must be in cache or not
• Whether the cache is with layout or not

See you Tomorrow

Like many other symfony features, the symfony cache sub-framework is very flexible and allows the developer to configure the cache at a very fine-grained level.

Tomorrow, we will talk about the last step of an application life-cycle: the deployment to the production servers.