Reproducible builds are a set of software development practices that create "a verifiable path from human readable source code to the binary code used by computers". In other words, if you don't change the source code, the compilation result should always be exactly the same.
Explained more simply in the case of Symfony: if you build the container and warm up the cache of the same unchanged application multiple times, the result should always be the same.
Why are reproducible builds important? Because software inspections and audits are done on the source code, but applications always run compiled on devices. If the build is reproducible, you can compile the source code that you audited and verify that the compiled result is exactly the same as the one being run on some device. Multiple parties can redo this process independently and ensure they all get exactly the same result, ensuring that the binary code comes from the given source code.
In practice, reproducible builds means that the compilation process must be completely deterministic. The compiled code can't contain date/time values or randomly generated values. That was not the case for Symfony and some of its major third-party dependencies like Monolog and Doctrine.
This required some changes in the way Symfony applications are compiled (we did those changes in the 3.4 branch, so you don't have to install Symfony 4 to use them):
- Variable names are no longer random (
uniqid(mt_rand(), true), false)
) in compiled Twig templates (twig#2621) - Class names generated for lazy services proxies are no longer random
(
hash('sha256', spl_object_hash($definition).$this->salt)
) in Symfony container (symfony#25978) - Monolog bundle no longer generates random IDs for some of its services
(
uniqid('monolog.gelf.publisher.', true)
) (monolog-bundle#248) - The Filesystem adapter of the Symfony Cache component now makes non-expirable items independent from time (before they expired in one year) (symfony#26127)
- The container compiled by Symfony uses the build datetime to generate the
unique hash of the container (it used a simple
time()
call). Now this time is configurable with thekernel.container_build_time
parameter (symfony#26128)
The work to make Symfony builds reproducible is finished on the Symfony side, but you may still face some issues in other dependencies commonly used in Symfony apps. For example:
- The APCu prefix generated by Composer's optimized autoloader is not deterministic (composer#7049)
The ProxyManager library is NOT about Doctrine proxies (which have deterministic names since years). It is about lazy service proxies (we made the class name deterministic in Symfony, but some property names have to be handled in the library).
And deterministic property names in ProxyManager is already a thing if you install their 2.2 release (which require PHP 7.2+). See https://github.com/Ocramius/ProxyManager/pull/385 for the PR. The discussion on their side seems to be that they don't want to backport this to older versions.
And for Composer, there is a way to get a deterministic build: using the authoritative class map optimization for the autoloader instead of the APCu one.
@Christophe thanks for reviewing this! I've removed the mention to ProxyManager.
@javier Thank you for the blog post. Probably worth mentioning the patch in https://github.com/Ocramius/ProxyManager/pull/385 for people who are not on 7.2 yet. They might want to apply it. If you want you could also link the demo project: https://github.com/lstrojny/symfony-reproducible-builds
I really like this changes, small changes could change much more than you think and with a diff it's easier to determind what really happens!
Very good topic. Thank you