Archives


Master Symfony2 fundamentals

Be trained by SensioLabs experts (2 to 6 day sessions -- French or English).
trainings.sensiolabs.com

Discover the SensioLabs Support

Access to the SensioLabs Competency Center for an exclusive and tailor-made support on Symfony
sensiolabs.com

Fabien Potencier
Dailymotion, powered by symfony
by Fabien Potencier – February 18, 2009 – 25 comments

Dailymotion

Dailymotion, one of the world's largest independent video entertainment website, is now powered by symfony. Dailymotion is among the top 60 websites worldwide. In December 2008, more than 41,9 million unique users visited the website and they viewed more than 922 million online videos (source: comScore, December 2008). Dailymotion is also the number 2 French site in the world.

Dailymotion

Founded in 2005, the Dailymotion website has always been powered by PHP and managed by a team of in-house developers. Over the years, a custom framework has been developed to power the platform.

But mid-2008, they decided to move away from their home-made framework and migrate to an Open-Source framework.

Dailymotion by the numbers

What to know all the nitty-gritty details of the Dailymotion architecture? Here are some interesting numbers:

  • 50 PHP machines serving about 5000 PHP pages per second
  • 17 DB machines
  • 30 memcache machines for a total of 210 Gb of memory storage
  • 60 Gb/s of bandwidth
  • 1.2 petabytes of storage

And that does not account for the machines that encode the videos, host the search engine, ... All in all, they have more than 430 machines!

Why migrate to an Open-Source framework?

Even if Dailymotion has more than 10 developers dedicated to the website development, they wanted to be more focused on their business, instead of being forced to maintain a framework.

Over time, they started to feel the pain of maintaining their own framework: the maintenance of the framework itself started to take way too much time: adding new features, documenting them, training new developers, keep everything coherent, etc...

By choosing an Open-Source framework with a large community, they free themselves from everything other than their business.

Olivier Poitrey, one of the co-founder and CTO of Dailymotion, sums up the main advantages of moving to a framework:

  • Don’t reinvent the wheel
  • Maintain less code
  • Better compliance to coding conventions
  • Implement new processes like unit/functional testing easily

Why symfony?

Thanks to the creativity of their marketing department, the Dailymotion developers add new features on a nearly everyday basis. So, rewriting the website from scratch was out of question. They just could not have afforded to stop all new developments, waiting for the new code base to be finished.

They needed a way to migrate step by step from the old code base to the new one, and keep adding new features at the same time. And symfony was the ideal candidate. Why?

First, because the symfony framework is one of the most well-known PHP frameworks, and one used by many high-traffic websites.

Then, the framework is well decoupled, meaning that each component can be used by itself, without the whole framework.

Eventually, the other big selling point of symfony was its flexibility. It is very easy to override the default behaviors or add new ones if needed. And that's exactly what they needed.

The migration process

The main challenge of the migration process was to avoid breaking too many things by introducing small doses of symfony.

The first change was to make symfony handle the request, dispatch it to the old code, get back the response content, and send back the response to the browser. Wrapping the existing code was made possible by using the symfony 2.0 sfRequestHandler class. We decided to use symfony 2.0 for its raw speed, even if it is not public yet.

By the way, that is the classic process of the symfony framework evolution. Sensio Labs uses symfony for its customers and a lot of new features are added based on their needs and feedback. For the Dailymotion migration, Sensio Labs helped Dailymotion define the migration strategy and the target architecture.

The next step was the migration of the mod_rewrite rules to the symfony routing. Thanks to the great flexibility of the Symfony routing, Dailymotion has been able to extend the default behavior to fit its unique needs: object parameters, labeled variables, segment grouping, arbitrary ordered optional segments, centralized formats, ...

Along the way, the Dailymotion team contributed back code to the symfony community. Some code has already been committed and is already available as part of the symfony 1.2 release. Some other enhancements are proposed as tickets.

One of the other big changes was the introduction of massive unit and functional tests. To migrate with confidence, more than 5000 unit and functional tests have been written, using lime and the symfony test browser.

During phase 1, the following components of symfony has been used:

Components used during phase 1

What's next?

Even if the Dailymotion website has only been powered by symfony for less than a week, the next migration steps are already planned:

  • Introduction of symfony controllers
  • Usage of symfony's facilities like sfI18n, sfUser, sfCookieSessionStorage, ...
  • Usage of symfony's 2.0 service container
  • Usage of symfony's model system (Doctrine)

Are you still using YouTube for your videos? I think it's time we all switch to Dailymotion ;)

Comments RSS

  • gravatar
    #1 avorobiev said on the 2009/02/18 at 20:12
    It's a great achievements!
    But why is Symfony 2.0 not public?
  • gravatar
    #2 fabien said on the 2009/02/18 at 20:14
    @avorobiev: because symfony 2.0 is in a pre-alpha stage, where we test a lot of things. As soon as the main elements will be more stable, it will be published. But keep in mind that most components in symfony 2.0 are just the same as in symfony 1.2: the request, the response, the user, the form, the routing, ...
  • gravatar
    #3 eko said on the 2009/02/18 at 20:36
    Great news !
    I'm happy to see that this kind of web services use symfony !
    A small step for the man, a big step for the humanity.
  • gravatar
    #4 Tonio said on the 2009/02/18 at 20:55
    Symfony 2.0 is supposed to use PHP 5.3, no ? So their server runs an unstable PHP version ?

    Good news for sf2, anyway ... seems that some parts are already quite stable.
  • gravatar
    #5 Craig Mason said on the 2009/02/18 at 20:56
    That's absolutely fantastic news.

    But it would be good to see SF 2.0 in SVN :)
  • gravatar
    #6 fabien said on the 2009/02/18 at 21:12
    @Tonio: As I said before, symfony 2.0 is still a moving target. And for now, it is still compatible with PHP 5.2.
  • gravatar
    #7 puentesdiaz said on the 2009/02/18 at 22:38
    Just i can say : GUAU !!!

    Do you imagine, when we can say microsoft.com is powered by Symfony :D:D:D
  • gravatar
    #8 tenshu said on the 2009/02/19 at 09:18
    My opinion is that is a weird way to announce symfony 2.0 to the community.
    Why no code and no specs are published?

    But i'll stick on the good news ...
  • gravatar
    #9 Cerbear said on the 2009/02/19 at 09:35
    +1 for the Great news!

    I'm happy to see the framework I choose, is also their choice ;)

    Keep up the good work!
  • gravatar
    #10 fabien said on the 2009/02/19 at 09:57
    @tenshu: it is not the first announce on symfony 2.0. I have talked about it at symfony camp 2007 and symfony camp 2008. Nothing new here.
  • gravatar
    #11 john said on the 2009/02/19 at 10:28
    well done symfony! great news.
  • gravatar
    #12 Mindphreaker said on the 2009/02/19 at 11:57
    Congratulation!
  • gravatar
    #13 halfer said on the 2009/02/19 at 12:25
    This is splendid news - it renews my enthusiasm for checking out the latest stuff. I shall look forward to seeing what things the team have cooked up for us in s2.0!

    Congratulations to the core team, and all sf contributors, for this excellent win.
  • gravatar
    #14 marijn said on the 2009/02/19 at 12:31
    This makes me even more excited about Symfony 2.0
  • gravatar
    #15 lloyd27 said on the 2009/02/19 at 12:40
    What is sfServiceContainer about?
  • gravatar
    #16 dereck said on the 2009/02/19 at 12:56
    @15lloyd27

    some dependency injection related stuff :)
  • gravatar
    #17 Fabien said on the 2009/02/19 at 13:04
    @lloyd27: read more about it here: http://fabien.potencier.org/talk/17/symfony-camp-2008-beyond-1-2
  • gravatar
    #18 lloyd27 said on the 2009/02/19 at 14:23
    Ok thanks :)
  • gravatar
    #19 Ross said on the 2009/02/20 at 21:54
    It is really excellent news that such a big site is making use of symfony. Perhaps we could get some more details about how the 2.0 changes are shaping up?
  • gravatar
    #20 Ben said on the 2009/02/21 at 18:54
    It's quite impressive, but i can see a few drawbacks to this news :
    1/ they use a non-released version of symfony.
    2/ they used consulting from sensio labs themselves (which is a web agency)
    3/ they haven't released any code.

    So, to sum it up, they're using something that nobody else can use, and they've been doing this work by paying a web agency.

    I'm not sure it's such a great day for the "open source" spirit of this project (which i find remarkable and use myself for my projects).
  • gravatar
    #21 halfer said on the 2009/02/21 at 22:56
    @Ben - I think we'll see the non-DM parts of the effort added into symfony soon. True, we don't get the code at the same time, but I think we will get it - which is still a huge open source benefit.

    Meanwhile this *is* great news, IMO, since it draws attention to the project and as a result may strengthen the size of the contributing community.
  • gravatar
    #22 Fabien said on the 2009/02/23 at 09:05
    @Ben:

    Some more information:

    1/ As I have already stated in the comments, and as shown in the post schema,
    the Dailymotion code is based on symfony 1.2 libraries (request, response,
    routing, cache, ...), which are all free to uses, free to download, and
    available today. The only code they have used from symfony 2.0 is the
    sfRequestHandler class which represents 100 PHP line of codes. And this class
    will be released as part of the symfony 2.0 version.

    2/3/ Sensio Labs does consulting for Dailymotion, helping them with the
    software architecture, but they write all the code. As far as contributions
    are concerned, they have already contributed code to the symfony routing
    system to make it more flexible. They have opened tickets with patches to add
    some new features to symfony. They have posted to the symfony developer
    mailing-list to suggest enhancements. And this is the spirit of the
    Open-Source philosophy. Not so many symfony share as much as they do. What
    would you expect? That they release their website code under an Open-Source
    license? Come on. Where can I download everything you have ever coded with
    symfony?

    Dailymotion uses symfony, like many others, and they don't have to release
    anything they have done with it. That's the power of the MIT license. But
    thanks to people like Dailymotion, and many others, Sensio Labs have some
    money to work on the symfony project, add new features, add documentation, and
    release everything under a free license, with no string attached. I think
    that's huge!
  • gravatar
    #23 Valere said on the 2009/02/23 at 17:46
    bravo !
  • gravatar
    #24 Colnector said on the 2009/03/01 at 16:03
    Indeed very good news! :)

    As I expect my symfony-powered website to grow quickly, I'm happy to learn that the framework will be able to take the heat.
  • gravatar
    #25 dway said on the 2009/03/21 at 01:20
    I'd switch to Dailymotion if they had an API :)