Jobeet - Day 22: The Cache

Previously on Jobeet

Today's tutorial starts the last week of Jobeet. We will talk about a very interesting topic: caching.

The symfony framework has many built-in cache strategies. For instance, the YAML configuration files are first converted to PHP and then cached on the filesystem. We have also seen that the modules generated by the admin generator are cached for better performance.

Today, we will talk about another cache: the HTML cache. To improve your website performance, you can cache whole HTML pages or just parts of them.

Creating a new Environment

By default, the template cache feature of symfony is enabled in the settings.yml configuration file for the prod environment, but not for the test and dev ones:

prod:
  .settings:
    cache: on
 
dev:
  .settings:
    cache: off
 
test:
  .settings:
    cache: off
 

As we need to test the cache feature before going to production, we can activate the cache for the dev environment or create a new environment. Recall that an environment is defined by its name (a string), an associated front controller, and optionally a set of specific configuration values.

To play with the cache system on Jobeet, we will create a cache environment, similar to the prod environment, but with the log and debug information available in the dev environment.

Create the front controller associated with the new cache environment by copying the dev front controller web/frontend_dev.php to web/frontend_cache/php:

// web/frontend_cache.php
if (!in_array(@$_SERVER['REMOTE_ADDR'], array('127.0.0.1', '::1')))
{
  die('You are not allowed to access this file. Check '.basename(__FILE__).' for more information.');
}
 
require_once(dirname(__FILE__).'/../config/ProjectConfiguration.class.php');
 
$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'cache', true);
sfContext::createInstance($configuration)->dispatch();
 

That's all there is to it. The new cache environment is now useable. The only difference is the second argument of the getApplicationConfiguration() method which is the environment name, cache.

You can test the cache environment in your browser by calling its front controller:

http://jobeet.localhost/frontend_cache.php/

The front controller script begins with a code that ensures that the front controller is only called from a local IP address. This security measure is to protect the front controller from being called on the production servers. We will talk about this in more details in tomorrow's tutorial.

For now, the cache environment inherits from the default configuration. Edit the settings.yml configuration file to add the cache environment specific configuration:

# apps/frontend/config/settings.yml
cache:
  .settings:
    error_reporting: <?php echo (E_ALL | E_STRICT)."\n" ?>
    web_debug:       on
    cache:           on
    etag:            off
 

In these settings, the symfony template cache feature has been activated with the cache setting and the web debug toolbar has been enabled with the web_debug setting.

As we also want to log SQL statements, we need to change the database configuration. Edit databases.yml and add the following configuration at the beginning of the file:

# config/databases.yml
cache:
  propel:
    class: sfPropelDatabase
    param:
      classname: DebugPDO
 

As the default configuration caches all settings in the cache, you need to clear it before being able to see the changes in your browser:

$ php symfony cc

Now, if your refresh your browser, the web debug toolbar should be present in the top right corner of the page, as it is the case for the dev environment.

Cache Configuration

The symfony template cache can be configured with the cache.yml configuration file. The default configuration for the application is to be found in apps/frontend/config/cache.yml:

default:
  enabled:     off
  with_layout: false
  lifetime:    86400
 

By default, as all pages can contain dynamic information, the cache is globally disabled (enabled: off). We don't need to change this setting, as we will enable the cache on a page by page basis.

The lifetime setting defines the server side life time of the cache in seconds (86400 equals to a day).

You can also work the other way around: enable the cache globally and then, disable it on specific pages that cannot be cached. It depends on which represents the less work for your application.

Page Cache

As the Jobeet homepage will probably be the most visited page of the website, instead of requesting the data from the database each time a user accesses it, it can be cached.

Create a cache.yml file for the sfJobeetJob module:

# plugins/sfJobeetJob/modules/sfJobeetJob/config/cache.yml
index:
  enabled:     on
  with_layout: true
 

The cache.yml configuration file have the same properties than other symfony configuration files like view.yml. It means for instance that you can enable the cache for all actions of a module by using the special all key.

If your refresh your browser, you will see that symfony has decorated the page with a box indicating that the content has been cached:

The box gives some precious information about the cache key for debugging, like the lifetime of the cache, and the age of it.

If you refresh the page again, the color of the box changed from green to yellow, indicating that the page has been retrieved from the cache:

Also notice that no database request has been made in the second case, as shown in the web debug toolbar.

Even if the language can be changed on a per-user basis, the cache still works as the language is embedded in the URL.

When a page is cacheable, and if the cache does not exist yet, symfony stores the response object in the cache at the end of the request. For all other future requests, symfony will send the cached response without calling the controller:

This has a great impact on performance as you can measure for yourself by using tools like JMeter.

An incoming request with GET parameters or submitted with the POST, PUT, or DELETE method will never be cached by symfony, regardless of the configuration.

The job creation page can also be cached:

# plugins/sfJobeetJob/modules/sfJobeetJob/config/cache.yml
new:
  enabled:     on
 
index:
  enabled:     on
 
all:
  with_layout: true
 

As the two pages can be cached with the layout, we have created a all section that defines the default configuration for the sfJobeetJob module.

Clearing the Cache

If you want to clear the page cache, you can use the cache:clear task:

$ php symfony cc

The cache:clear task clears all the symfony caches stored under the main cache/ directory. It also takes options to selectively clear some parts of the cache. To only clear the template cache for the cache environment, use the --type and --env options:

$ php symfony cc --type=template --env=cache

Instead of clearing the cache each time you make a change, you can also disable the cache by adding any query string to the URL, or by using the "Ignore cache" button from the web debug toolbar:

Action Cache

Sometimes, you cannot cache the whole page in the cache, but the action template itself can be cached. Put another way, you can cache everything but the layout. Change the configuration for the index action cache accordingly:

# plugins/sfJobeetJob/modules/sfJobeetJob/config/cache.yml
index:
  enabled:     on
  with_layout: false
 

By changing the with_layout setting to false, you have disabled layout caching.

Clear the cache:

$ php symfony cc

Refresh your browser to see the difference:

Even if the flow of the request is quite similar in the simplified diagram, caching without the layout is much more resource intensive.

Partial and Component Cache

For highly dynamic websites, it is sometimes impossible to cache the whole action template. For those cases, you need a way to configure the cache at the finer-grained level. Thankfully, partials and components can be cached.

Let's cache the language component by creating a cache.yml file for the sfJobeetLanguage module:

# plugins/sfJobeetJob/modules/sfJobeetLanguage/config/cache.yml
_language:
  enabled: on
 

Configuring the cache for a partial or a component is as simple as adding an entry with its name. The with_layout option is not taken into account for this type of cache as it does not make any sense:

Contextual or not?

The same component or partial can be used in many different templates. The job list partial for instance is used in the job and category modules. As the rendering is always the same, the partial does not depend on the context in which it is used and the cache is the same for all templates (the cache is still obviously different for a different set of parameters).

But sometimes, a partial or a component output is different, based on the action in which it is included (think of a blog sidebar for instance, which is slightly different for the homepage and the blog post page). In such cases the partial or component is contextual, and the cache must be configured accordingly by setting the contextual option to true:

_sidebar:
  enabled:    on
  contextual: true
 

You can now revert the with_layout setting to true as it makes more sense for the Jobeet website.

Forms in Cache

Storing the job creation page in the cache is problematic as it contains a form. To better understand the problem, go to the "Post a Job" page in your browser to seed the cache. Then, clear your session cookie, and try to submit a job. You must see an error message alerting you of a "CSRF attack":

Why? As we have configured a CSRF secret when we created the frontend application, symfony embeds a CSRF token in all forms. To protect you against CSRF attacks, this token is unique for a given user and for a given form.

The first time the page is displayed, the generated HTML form is stored in the cache with the current user token. If another user comes afterwards, the page from the cache will be displayed with the first user CSRF token. When submitting the form, the tokens do not match, and an error is thrown.

How can we fix the problem as it seems legitimate to store the form in the cache? The job creation form does not depend on the user, and it does not change anything for the current user. In such a case, no CSRF protection is needed, and we can remove the CSRF token altogether:

// plugins/sfJobeetJob/lib/form/JobeetJobForm.class.php
class JobeetJobForm extends BaseJobeetJobForm
{
  public function __construct(BaseObject $object = null, $options = array(), $CSRFSecret = null)
  {
    parent::__construct($object, $options, false);
  }
 
  // ...
}
 

After doing this change, clear the cache and re-try the same scenario as above to prove it works as expected now.

The same configuration must be applied to the language form as it is contained in the layout and will be stored in the cache. As the default sfLanguageForm is used, instead of creating a new class, just to remove the CSRF token, let's do it from the action and component of the sfJobeetLanguage module:

// plugins/sfJobeetJob/modules/sfJobeetLanguage/actions/components.class.php
class sfJobeetLanguageComponents extends sfComponents
{
  public function executeLanguage(sfWebRequest $request)
  {
    $this->form = new sfFormLanguage($this->getUser(), array('languages' => array('en', 'fr')));
    unset($this->form[$this->form->getCSRFFieldName()]);
  }
}
 
// plugins/sfJobeetJob/modules/sfJobeetLanguage/actions/actions.class.php
class sfJobeetLanguageActions extends sfActions
{
  public function executeChangeLanguage(sfWebRequest $request)
  {
    $form = new sfFormLanguage($this->getUser(), array('languages' => array('en', 'fr')));
    unset($form[$this->form->getCSRFFieldName()]);
 
    // ...
  }
}
 

The getCSRFFieldName() returns the name of the field that contain the CSRF token. By unsetting this field, the widget and the associated validator are removed.

Removing the Cache

Each time a user posts and activates a job, the homepage must be refreshed to list the new job.

As we don't need the job to appear in real-time on the homepage, the best strategy is to lower the cache life time to something acceptable:

# plugins/sfJobeetJob/modules/sfJobeetJob/config/cache.yml
index:
  enabled:  on
  lifetime: 600
 

Instead of the default configuration of one day, the cache for the homepage will be automatically removed every ten minutes.

But if you want to update the homepage as soon as a user activates a new job, edit the executePublish() method of the sfJobeetJob module to add manual cache cleaning:

// plugins/sfJobeetJob/modules/sfJobeetJob/actions/actions.class.php
public function executePublish(sfWebRequest $request)
{
  $request->checkCSRFProtection();
 
  $job = $this->getRoute()->getObject();
  $job->publish();
 
  if ($cache = $this->getContext()->getViewCacheManager())
  {
    $cache->remove('sfJobeetJob/index?sf_culture=*');
    $cache->remove('sfJobeetCategory/show?id='.$job->getJobeetCategory()->getId());
  }
 
  $this->getUser()->setFlash('notice', sprintf('Your job is now online for %s days.', sfConfig::get('app_active_days')));
 
  $this->redirect($this->generateUrl('job_show_user', $job));
}
 

The cache is managed by the sfViewCacheManager class. The remove() method removes the cache associated with an internal URI. To remove cache for all possible parameters of a variable, use the * as the value. The sf_culture=* we have used in the code above means that symfony will remove the cache for the English and the French homepage.

As the cache manager is null when the cache is disabled, we have wrapped the cache removing in an if block.

The sfContext class

The sfContext object contain references to symfony core objects like the request, the response, the user, and so on. As sfContext acts like a singleton, you can use the sfContext::getInstance() statement to get it from anywhere and then have access to any symfony core objects:

$user = sfContext::getInstance()->getUser();
 

You can even use sfContext as a registry and add your own objects using the set() methods. It takes a name and an object as arguments and the get() method can be used later on to retrieve an object by name:

sfContext::getInstance()->set('job', $job);
$job = sfContext::getInstance()->get('job');
 

Testing the Cache

As we have made a lot of changes to the cache configuration, here is the one you need to have for the job module:

# plugins/sfJobeetJob/modules/sfJobeetJob/config/cache.yml
index:
  enabled:     on
 
new:
  enabled:     on
 
all:
  with_layout: true
 

Before starting, we need to change the configuration for the test environment to enable the cache layer:

# apps/frontend/config/settings.yml
test:
  .settings:
    error_reporting: <?php echo ((E_ALL | E_STRICT) ^ E_NOTICE)."\n" ?>
    cache:           on
    web_debug:       off
    etag:            off
 

Let's test the job creation page:

// test/functional/frontend/jobActionsTest.php
$browser->
  info('  7 - Job creation page')->
 
  get('/fr/')->
  with('view_cache')->isCached(true, true)->
 
  createJob(array('category_id' => $browser->getProgrammingCategory()->getId()), true)->
 
  get('/fr/')->
  with('view_cache')->isCached(true, true)->
  with('response')->checkElement('.category_programming .more_jobs', '/29/')
;
 

The view_cache tester is used to test the cache. The isCached() method takes two booleans:

• Whether the page must be in cache or not
• Whether the cache is with layout not not

Even with all the tools provided by the functional test framework, it is sometimes easier to diagnose problems within the browser. It is quite easy to accomplish. Just create a front controller for the test environment. The logs stored in log/frontend_test.log can also be very helpful.

See you Tomorrow

Like many other symfony features, the symfony cache sub-framework is very flexible and allows the developer to configure the cache at a very fine-grained level.

Tomorrow, we will talk about the last step of an application life-cycle: the deployment to the production servers.