A few days ago we unveiled the redesign of the Symfony Documentation website. In addition to the new visual design, we revamped all the internals used to build the docs from their source. This article explains how we did it.
Building Symfony Docs
The documentation of the Symfony project is Open Source and managed using the github.com/symfony/symfony-docs repository. Thousands of people from all over the world have contributed to these docs. It's one of the most popular doc projects of the entire Open Source space.
Symfony Docs use reStructuredText (or RST for short) format to write its contents. Then, we use Sphinx, a documentation building tool, to transform those RST files into the HTML contents served on our website.
This setup has served us well for some years, but it had some serious problems for us. First, Sphinx and most of the tools related to RST are written in Python programming language, which we don't know well. Second, updating and maintaining Sphinx caused many troubles for us, because it's too strict and unforgiving.
That's why around 2019 we started to seriously look for an alternative based on PHP.
Sticking to RST format
An obvious solution would have been to switch Symfony Docs to Markdown, the most popular doc format. This was quickly discarded because of the colossal amount of contents of Symfony Docs (converting all of them to Markdown would take us forever).
Before continuing, let's play a game: can you guess the size (in lines, words and characters) of the entire Symfony Docs? I'll show you the answer at the end of this blog post.
Apart from the contents size, there was another problem. Markdown is great for simple contents, but it's not that great for complex technical docs. RST provides all the features that you'll need when writing complex docs, such as cross references to internal doc sections and programmable "directives" to create custom content blocks. In Markdown it's impossible to do that or you need to use hacks which aren't part of the standard.
Building a RST parser in PHP
Building a doc parser is not a simple feat, much less for a full-featured format like RST. Luckily, Symfony was not the only PHP-related project that needed a RST parser. Doctrine project uses RST for its docs too and also wanted to get rid of Sphinx to build their docs.
That's why some Doctrine people created (and open sourced) the doctrine/rst-parser project based on a previous but incomplete RST parser. It's not a fully-compliant RST parser, but it's super fast, easy to use and covers all our RST needs.
The RST parser problem was now solved. Next, we needed to create a full "doc builder" to replace Sphinx.
Creating a Doc Builder in PHP
Thanks to the flexibility of Doctrine's RST parser, we could build an application on top of it called symfony-tools/doc-builder. In short, this tool finds the RST files, parses them into HTML (including code highlighting), and saves them in JSON files with a certain structure expected by symfony.com.
This doc builder also defines and processes all the custom RST directives used in Symfony Docs. For example, when you browse the Symfony routing article, you can select the format of certain code blocks (annotations, attributes, YAML, XML, etc.) This is not part of the RST standard but, contrary to Markdown, in RST there's a standard way of extending the format.
That's why Symfony Docs use a proprietary .. configuration-block: directive
which is processed in the ConfigurationBlockDirective class of our doc builder.
Updating Doc Internals in symfony.com
All the necessary pieces to replace Sphinx/Python with our own PHP Doc Builder were ready. The last step was to update symfony.com code to make use of it. This took us a bit longer than expected because of the "technical debt" accumulated during the past years.
Thankfully we had some PHPUnit tests in place to check that the docs worked as expected (e.g. expected page titles and table-of-contents, doc navigation, etc.) This gave us some confidence before starting this big refactoring.
We ended up deleting most of the existing code related to docs and replaced it with some DTOs created from the JSON files generated by the doc builder. These DTOs contain all the data needed to fully render a page (its table-of-contents items, the locale of the contents, the other Symfony versions in which the page is published, etc.) This helped us remove all the logic from controllers and templates.
Sphinx and Python were now gone. The RST parser and the Doc Builder had been fully integrated in symfony.com. What was next? Redesigning the doc section!
Redesigning the Website
A few months ago we unveiled the new Symfony Bundles Doc section. This was the first symfony.com section to use the new RST parser and Doc Builder. We also created a unique design for it, completely different to the rest of the site.
From the very beginning, this bundles section was a test ground for the Symfony Docs redesign. It allowed us to test the new doc building for real, to perform some experiments and to get feedback from some members of the community. We used all that to build the new Symfony Docs website.
The main ideas behind the new design were:
- Overall design should be simpler, with less "noise"
- There should be more "breathing space" between contents (pages are wider now)
- Readability should be the top priority (larger and better typography)
Here's a comparison between the old (left) and the new (right) index page (click to make it larger):
Here's a screenshot of a typical book page under the new design (click to make it larger):
The new design is full of little details that help improve readability. It's also fully responsive and provides a dark mode alternative. The default mode is the same as your operating system. If you want to set it to light or dark explicitly, use the UI element located in the footer of all pages.
Thanks to All Who Made this Possible
The new Symfony Docs website is a good example of the advantages of Open Source in practice. We leveraged (and contributed to) different projects created and maintained by different organizations. The result is a PHP doc builder which is faster and much easier to maintain than the previous non-PHP builder.
Thanks to all contributors who made this possible:
Did you guess the Symfony Docs size?
At the beginning of this article I asked you to try to guess the size of the
Symfony Docs contents. Here's the answer: as of October 2021, and according to
wc command, the entire Symfony Docs contents across its 25 versions (from 2.0
to 6.0) and including all code examples, comprises 2,712,374 lines,
9,787,338 words and 97,698,319 characters.
To put things in perspective, the "Lord of the Rings" trilogy contains 576,459 words, making Symfony Docs 17 times bigger.
Is there a way to get the old design instead, like a secret cookie or something?
Hi, Is it possible to make the background of code blocks darker than the background of the page ? (like in light mode, the background of code blocks is dark) :)
Lionel, sorry there's no way to use the old design.
Christophe, we're discussing about that internally. We might do some changes related to that.
The new design is a huge boon to readability. It looks smart & modern now, casting off a somewhat dated look. Most importantly, it makes documentation feel simpler and more organized. Bravo.
Yes, please change the background color of the code blocks.
Nice work, however the layout on 2 columns hinders the readability a bit. The sponsor part takes up too much space and makes reading more difficult.
I'm so disappointed readability is fucked, old design is mush better: it feels like a huge wall of text if you're quickly skimming over it. We never had the problem with the old layout because code snippets were clearly identified as a separate block. The sponsor part takes up too much space. The documentation is the home for the developers, how does not care about sponsors.
You should really think about it and listen to the community who contributed to the documentation.
We've made some changes and tweaks to the design based on community feedback. One of those changes was to add a subtle background to code blocks.
Regarding sponsors, they help fund the Open Source Symfony project, so we need to highlight them. Thanks for understanding.
I agree with the others, even after the change the code blocks are to bright. Before it was easier to identify the code blocks which made scrolling through the docs to find a certain example more comfortable. I would also prefer to have the TOC in a sticky menu, so navigating on the page is more comfortable.
But I don't mind the sponsors in a prominent place, they help sponsoring symfony which should be rewarded.
Besides those minor things i like the changes. Thank you very much for continuously improving things!
The sponsors are dominating the documentation. On the whole page the sidebar and the sponsors are placed on the left, except the documentation.
And why you dont combine something useful with that sponsors, like the top menu, so that the sidebar is not only ads, but also useful for the users.
The fontsize is also way too big on desktop, but thats fine cause of the browser zoom out.
cool
I thought I was crazy, but then I saw comments from other users. :) Using the new documentation is inconvenient for some reason which is hard to explain. I just have problems with focus. Headings, text... they are not where they should be. Scrolling this part of screen is also strange. I did some changes in DOM model and CSS to make this usable and viola - it worked. So: I agree with opinion that sponsors sticky column should be removed. It's perfect place to put anchor links (if screen resolution will allow). If not - then you may also consider moving this sticky sponsors to the left. They are now overwhelming. Anyway - FullHD resolution is not uncommon these days, so i believe there is a way to make documentation more friendly.
please please change the code block backgroud to the old one, then centrally aligned theme of the old one was easy on the eye n could find what u were looking for easily this is a mess with the partner logs on the right -- very distracting
please revert it back
Overall design should be simpler, with less "noise" There should be more "breathing space" between contents (pages are wider now) Readability should be the top priority (larger and better typography)
-- there is more distraction -- Cant concentrate on the code blocks -- Readability received no priority -- Why u did this ?? is the question - i understant the change to internals but design .... didnt achieve the goal at all
if want to change it than going the route of symfonycasts tutorial page for e.g. https://symfonycasts.com/screencast/symfony/setup would be better
at least hightlight functions, variables, keywords, differently same as IDEs
Main issue with new design from my point of view is where center of the screen (on large display) is located relatively to the text content.
In old design (like the page with this article) aside is at the left, so center of the screen is close to the start of the text lines of main article. Taking is mind that we're reading English from left to right it means that eyes are going from the center (neutral position) to the right and it is comfortable.
In new design aside is at the right, making neutral position of the eyes to point them to the end of line. Because of this eyes needs to move quite far to the left for each line and then get back to neutral position. It feels inconvenient.
Simple swap of aside to the left side brings much better comfort to the eyes.
Besides of this it looks strange that title of the article and version switch are not sticky.
Please also notice that one-line annotation block is a bit narrower then expected by design effectively resulting into smaller offset from the icon to the bottom of the block. Example can be seen here: https://symfony.com/doc/current/setup.html#installing-packages
I have to echo some of the other sentiments here. There needs to be contrast in the code blocks. Go back to black code blocks with bright code text. At the moment there is almost no contrast between code and comments in the code and being on a light background makes it worse.
Also there is hardly any distinction between the headers. h2 is 4px off h3. At that size (32px) they apear the same. There is not enough segrigation as well. A bit more vertical white space would go a long way to solving this.
As it stands at the moment it's visually awkward.
One of the great advantages of this redesign for RTL languages is the easy use of Google Translator on the page so that the page does not get cluttered.
I totally agree with Alexander Grimalovsky. My screen is 1366x768 and it's extremely uncomfortable to have to look all the way to left. I think it would be better the sponsors to be on the left and the content on the right. The old design was really great and comfortable to read.
Anyways, long live to Symfony and huge thanks to all the comunity and sponsors! :D
Wohoo. Well done!
I also agree with Alexander Grimalovsky, I find it uncomfortable to read as it is.
Thanks for everything you're doing though!