Dailymotion, powered by symfony
Dailymotion, one of the world's largest independent video entertainment website, is now powered by symfony. Dailymotion is among the top 60 websites worldwide. In December 2008, more than 41,9 million unique users visited the website and they viewed more than 922 million online videos (source: comScore, December 2008). Dailymotion is also the number 2 French site in the world.
Founded in 2005, the Dailymotion website has always been powered by PHP and managed by a team of in-house developers. Over the years, a custom framework has been developed to power the platform.
But mid-2008, they decided to move away from their home-made framework and migrate to an Open-Source framework.
Dailymotion by the numbers
What to know all the nitty-gritty details of the Dailymotion architecture? Here are some interesting numbers:
- 50 PHP machines serving about 5000 PHP pages per second
- 17 DB machines
- 30 memcache machines for a total of 210 Gb of memory storage
- 60 Gb/s of bandwidth
- 1.2 petabytes of storage
And that does not account for the machines that encode the videos, host the search engine, ... All in all, they have more than 430 machines!
Why migrate to an Open-Source framework?
Even if Dailymotion has more than 10 developers dedicated to the website development, they wanted to be more focused on their business, instead of being forced to maintain a framework.
Over time, they started to feel the pain of maintaining their own framework: the maintenance of the framework itself started to take way too much time: adding new features, documenting them, training new developers, keep everything coherent, etc...
By choosing an Open-Source framework with a large community, they free themselves from everything other than their business.
Olivier Poitrey, one of the co-founder and CTO of Dailymotion, sums up the main advantages of moving to a framework:
- Don't reinvent the wheel
- Maintain less code
- Better compliance to coding conventions
- Implement new processes like unit/functional testing easily
Thanks to the creativity of their marketing department, the Dailymotion developers add new features on a nearly everyday basis. So, rewriting the website from scratch was out of question. They just could not have afforded to stop all new developments, waiting for the new code base to be finished.
They needed a way to migrate step by step from the old code base to the new one, and keep adding new features at the same time. And symfony was the ideal candidate. Why?
First, because the symfony framework is one of the most well-known PHP frameworks, and one used by many high-traffic websites.
Then, the framework is well decoupled, meaning that each component can be used by itself, without the whole framework.
Eventually, the other big selling point of symfony was its flexibility. It is very easy to override the default behaviors or add new ones if needed. And that's exactly what they needed.
The migration process
The main challenge of the migration process was to avoid breaking too many things by introducing small doses of symfony.
The first change was to make symfony handle the request, dispatch it to the
old code, get back the response content, and send back the response to the
browser. Wrapping the existing code was made possible by using the symfony 2.0
sfRequestHandler class. We decided to use symfony 2.0 for its raw speed,
even if it is not public yet.
By the way, that is the classic process of the symfony framework evolution. Sensio Labs uses symfony for its customers and a lot of new features are added based on their needs and feedback. For the Dailymotion migration, Sensio Labs helped Dailymotion define the migration strategy and the target architecture.
The next step was the migration of the mod_rewrite rules to the symfony routing. Thanks to the great flexibility of the Symfony routing, Dailymotion has been able to extend the default behavior to fit its unique needs: object parameters, labeled variables, segment grouping, arbitrary ordered optional segments, centralized formats, ...
Along the way, the Dailymotion team contributed back code to the symfony community. Some code has already been committed and is already available as part of the symfony 1.2 release. Some other enhancements are proposed as tickets.
One of the other big changes was the introduction of massive unit and functional tests. To migrate with confidence, more than 5000 unit and functional tests have been written, using lime and the symfony test browser.
During phase 1, the following components of symfony has been used:
Even if the Dailymotion website has only been powered by symfony for less than a week, the next migration steps are already planned:
- Introduction of symfony controllers
- Usage of symfony's facilities like
- Usage of symfony's 2.0 service container
- Usage of symfony's model system (Doctrine)
Are you still using YouTube for your videos? I think it's time we all switch to Dailymotion ;)
Help the Symfony project!
As with any Open-Source project, contributing code or documentation is the most common way to help, but we also have a wide range of sponsoring opportunities.
Comments are closed.
To ensure that comments stay relevant, they are closed for old posts.
But why is Symfony 2.0 not public?
I'm happy to see that this kind of web services use symfony !
A small step for the man, a big step for the humanity.
Good news for sf2, anyway ... seems that some parts are already quite stable.
But it would be good to see SF 2.0 in SVN :)
Do you imagine, when we can say microsoft.com is powered by Symfony :D:D:D
Why no code and no specs are published?
But i'll stick on the good news ...
I'm happy to see the framework I choose, is also their choice ;)
Keep up the good work!
Congratulations to the core team, and all sf contributors, for this excellent win.
some dependency injection related stuff :)
1/ they use a non-released version of symfony.
2/ they used consulting from sensio labs themselves (which is a web agency)
3/ they haven't released any code.
So, to sum it up, they're using something that nobody else can use, and they've been doing this work by paying a web agency.
I'm not sure it's such a great day for the "open source" spirit of this project (which i find remarkable and use myself for my projects).
Meanwhile this *is* great news, IMO, since it draws attention to the project and as a result may strengthen the size of the contributing community.
Some more information:
1/ As I have already stated in the comments, and as shown in the post schema,
the Dailymotion code is based on symfony 1.2 libraries (request, response,
routing, cache, ...), which are all free to uses, free to download, and
available today. The only code they have used from symfony 2.0 is the
sfRequestHandler class which represents 100 PHP line of codes. And this class
will be released as part of the symfony 2.0 version.
2/3/ Sensio Labs does consulting for Dailymotion, helping them with the
software architecture, but they write all the code. As far as contributions
are concerned, they have already contributed code to the symfony routing
system to make it more flexible. They have opened tickets with patches to add
some new features to symfony. They have posted to the symfony developer
mailing-list to suggest enhancements. And this is the spirit of the
Open-Source philosophy. Not so many symfony share as much as they do. What
would you expect? That they release their website code under an Open-Source
license? Come on. Where can I download everything you have ever coded with
Dailymotion uses symfony, like many others, and they don't have to release
anything they have done with it. That's the power of the MIT license. But
thanks to people like Dailymotion, and many others, Sensio Labs have some
money to work on the symfony project, add new features, add documentation, and
release everything under a free license, with no string attached. I think
As I expect my symfony-powered website to grow quickly, I'm happy to learn that the framework will be able to take the heat.