Cache strategies

[mnapoli]

I am collecting stuff from various discussions so that we can collect all that in a single place and keep it up to date.

• Strategy A (Implement strategy A #32):
  • if the cache is empty, use /tmp
  • if the cache is not empty, use var/cache for the compiled container and /tmp for everything else
• Strategy B (master branch):
  • if the cache is empty, use /tmp
  • if the cache is not empty, copy it to /tmp
• Strategy C:
  • if the cache is empty, use /tmp
  • if the cache is not empty, symlink it to /tmp
• Strategy D:
  • if the cache is empty, use /tmp
  • if the cache is not empty, use var/cache for the compiled container and the system pool, and /tmp for other cache pools
  • that will trigger warnings in the logs

Benchmarks

Benchmark:	@Nyholm	@Nemo64
Strategy A
Strategy B
Strategy C
Strategy D

Please feel free to update the description of this issue with benchmark results or new strategies.

[mnapoli]

I have described the different strategies as best as I could, but I'm not really sure everything is correct. Feel free to correct it.

[Nyholm]

What is implemented in master is "copy 'pools' and symlink everything else". (Which is a really good "it works out of the box").

If you want optimal performance, you symlink everything. (Which is described in #28)

Here are some numbers. (I write them here to avoid edit conflicts).

Strategy C:
First request: 1800ms
First request with BrefKernel: 430ms

Strategy E: (current master, copy only "pools" and symlink rest)
First request: 1800ms
First request with BrefKernel: 510ms

[Nemo64]

I'm a bit confused about the strategies.

We need to be careful because the cache not only consists of the Container and pools, but also of templates, translations, generated doctrine proxies, the annotations file and the router by default. Everything I listed is perfectly warmable except the "pools" folder, which is completely configurable if you know how (which was my end result in #21). But we should also consider caches that don't exist by default but though other bundles. I don't know of any (I haven't searched) but the current master addresses this by allowing you to change which folders must be writable (#17).

I only see 4 Strategies (maybe 5 but the last one isn't implemented).

• Strategy E (as @Nyholm called it) which is the current master default and a combination of @mnapoli B/C in which everything possible is symlinked except for the pools folder because it should to be writable.
• Strategy C which is also possible with the master (see the readme pull request) but will trigger warnings because symlinking the pools folder makes read-only.
• Strategy D is similar to C but does not require the Bref Kernel. This is basically deploying a smyfony app without modification.
• Strategy ... F which is what I described in Symfony's capability to run on read-only out of the box #21 (comment) which requires 3 config file lines but also no Bref Kernel. In this case everything but the pools folder is deployed and pools (including system) is configured to /tmp without copying or warming it. In my quick test in that issue I noticed that not warming the pools folder at all is initially faster than copying it. But it still allows to write to it which strategy E and C does not which is probably an advantage on later requests.
• Strategy G (hypothetical) which i described in Symfony's capability to run on read-only out of the box #21 (comment) in which you'd use a custom cache adapter that is aware of the read-only situations and uses /tmp only for differences. This should be the ideal as there is no initial overhead (which is the only disadvantage of E) but the cache pool is still writable (which is the advantage of E and F)

We could try to build Strategy B (I did in #6) but it is just a worse variant of E so I don't see the point.

So let me try to put it in a table:

	Strategy E (master)	Strategy C	Strategy D	Strategy F	Strategy G
no init overhead	❌	~✔	✔	✔	✔
allows pool warming	✔	✔	✔	❌	✔
writable at runtime	✔	❌	❌	✔	✔
no warnings in log	✔	❌	❌	✔	✔
no bref kernel needed	❌	❌	✔	✔	~✔
doesn't break without warmup	✔	✔	❌	❌	❌
exists atm	✔	✔	✔	✔	❌

I consider "no bref kernel" an advantage since it's still not pretty to overwrite kernel methods. Strategy G does need a custom implementation but that could be a bundle or even just a library that isn't even specific to bref but can be used by anyone who has a symfony in a read only environment.

I'm not in the mood of benchmarking my project right now, maybe later, but I think I gave a good overview.

From my knowledge
I like E for being the most flexible and I'd generally recommend it if you don't want to spend time figuring out what you are doing. It should work with every project without configuration change.
Strategy F is the one I switched to for my projects because I think it is more elegant and the overhead of generating the annotation cache (the only pool cache that is properly warmed) is fairly small.
Strategy G is hypothetical atm but will probably be the best if you warm the cache.

[mnapoli]

Really great stuff here, thanks for all you details. I will write a proper response (I prefer to take the time to understand everything that you wrote).

In the meantime, I had another look at the root of the problem and found symfony/symfony#23354. I opened symfony/symfony#36515 to try moving things forward for the next major versions of Symfony.

[mnapoli]

Closing because this is a very old issue.