Skip to content

Commit ff42df5

Browse files
aerniMichael Aerni
andauthored
Rewrite Zero Downtime Deployments Docs (#1935)
Co-authored-by: Michael Aerni <michael@community.ch>
1 parent dbaf130 commit ff42df5

1 file changed

Lines changed: 253 additions & 55 deletions

File tree

content/collections/tips/zero-downtime-deployments.md

Lines changed: 253 additions & 55 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,15 @@
22
id: 5e1dbeb6-b59d-4c6c-a3fa-950c4372acba
33
blueprint: tips
44
title: 'Zero Downtime Deployments'
5-
intro: 'How zero-downtime deploy tools structure releases, and which Statamic paths should live outside the release folder.'
5+
intro: 'How zero-downtime deploy tools structure releases, and how to set up Statamic so the Stache cache and Git Automation work correctly with them.'
66
template: page
77
categories:
88
- development
99
- troubleshooting
1010
---
1111
## Understanding the folder structure
1212

13-
Zero downtime deployment services like [Laravel Forge](https://forge.laravel.com/), [Envoyer](https://envoyer.io/) and [Deployer](https://deployer.org/) typically use a multiple-release directory structure and symlinks to handle deployments.
13+
Zero downtime deployment services like [Laravel Forge](https://forge.laravel.com/), [Envoyer](https://envoyer.io/), [Ploi](https://ploi.io/) and [Deployer](https://deployer.org/) typically use a multiple-release directory structure and symlinks to handle deployments.
1414

1515
For example, with Laravel Forge:
1616

@@ -35,16 +35,19 @@ Every deployment has its own timestamped release directory, with a fresh clone o
3535
After a successful deployment, the `current` folder is then symlinked to the latest release. This symlink swap is the secret sauce for zero downtime.
3636

3737
## Cache storage
38+
3839
Statamic's content management heavily relies on caching, and sometimes it's necessary for the [Stache](/stache) to store absolute file paths in your app's cache. This can lead to deployment errors when users are hitting your frontend, since each release [exists in a separate timestamped folder](#understanding-the-folder-structure).
3940

4041
The solution is simple. Just as you should never share a cache between different websites, you should never share a cache between your deployed releases.
4142

4243
### How to avoid sharing file cache
4344

44-
There are two ways to avoid sharing a file cache between your deployment releases:
45+
There are three ways to avoid sharing a file cache between your deployment releases:
4546

4647
1. Some services, like Laravel Forge, may allow you to configure the "shared paths" between deployments. If your application allows for it, you could remove the `storage` directory from your site's shared paths, ensuring each release has its own `storage` folder.
48+
4749
2. Another option is to create a `cache` folder at the top level of your app, bypassing the shared `storage` folder. Configure your app to use a custom cache store location by changing `stores.file.path` in `config/cache.php`:
50+
4851
```php
4952
'stores' => [
5053
'file' => [
@@ -55,80 +58,231 @@ There are two ways to avoid sharing a file cache between your deployment release
5558
],
5659
```
5760

61+
This affects all file cache usage (rate limits, locks, queue locks, etc.), not just the Stache.
62+
63+
3. For a more targeted variant, give the Stache its own dedicated cache store. Laravel's default cache stays put. In `config/cache.php`, add a new store:
64+
65+
```php
66+
'stores' => [
67+
'stache' => [ // [tl! ++]
68+
'driver' => 'file', // [tl! ++]
69+
'path' => base_path('stache'), // [tl! ++]
70+
'lock_path' => base_path('stache'), // [tl! ++]
71+
], // [tl! ++]
72+
],
73+
```
74+
75+
Then in `config/statamic/stache.php`, point the Stache at the new store:
76+
77+
```php
78+
'cache_store' => 'stache', // [tl! ++]
79+
```
80+
5881
### How to avoid sharing Redis cache
5982

60-
To avoid sharing a Redis cache between your deployment releases, we recommend setting a cache prefix unique to each release on your filesystem. This can be configured by adding a `redis.cache.options.prefix` in `config/database.php`:
83+
There are two ways to avoid sharing a Redis cache between your deployment releases:
6184

62-
```php
63-
'redis' => [
64-
'cache' => [
65-
'url' => env('REDIS_URL'),
66-
'host' => env('REDIS_HOST', '127.0.0.1'),
67-
'password' => env('REDIS_PASSWORD'),
68-
'port' => env('REDIS_PORT', '6379'),
69-
'database' => env('REDIS_CACHE_DB', '1'),
70-
'options' => [ // [tl! ++]
71-
'prefix' => basename(base_path()).'_', // [tl! ++]
85+
1. Set a cache prefix unique to each release on your filesystem by adding a `redis.cache.options.prefix` in `config/database.php`:
86+
87+
```php
88+
'redis' => [
89+
'cache' => [
90+
'url' => env('REDIS_URL'),
91+
'host' => env('REDIS_HOST', '127.0.0.1'),
92+
'password' => env('REDIS_PASSWORD'),
93+
'port' => env('REDIS_PORT', '6379'),
94+
'database' => env('REDIS_CACHE_DB', '1'),
95+
'options' => [ // [tl! ++]
96+
'prefix' => basename(base_path()).'_', // [tl! ++]
97+
], // [tl! ++]
98+
],
99+
],
100+
```
101+
102+
This affects all Redis cache usage (rate limits, locks, queue locks, etc.), not just the Stache.
103+
104+
2. For a more targeted variant, give the Stache its own dedicated Redis cache store. Laravel's default Redis cache stays shared between releases, which is what you want for things like rate limits and queue locks.
105+
106+
In `config/database.php`, add a new Redis connection with a per-release prefix:
107+
108+
```php
109+
'redis' => [
110+
// ...
111+
'stache' => [ // [tl! ++]
112+
'url' => env('REDIS_URL'), // [tl! ++]
113+
'host' => env('REDIS_HOST', '127.0.0.1'), // [tl! ++]
114+
'password' => env('REDIS_PASSWORD'), // [tl! ++]
115+
'port' => env('REDIS_PORT', '6379'), // [tl! ++]
116+
'database' => env('REDIS_CACHE_DB', '1'), // [tl! ++]
117+
'options' => [ // [tl! ++]
118+
'prefix' => basename(base_path()).'_stache_', // [tl! ++]
119+
], // [tl! ++]
72120
], // [tl! ++]
73121
],
74-
],
75-
```
122+
```
76123

77-
## Git Automation
124+
In `config/cache.php`, add a cache store using that connection:
125+
126+
```php
127+
'stores' => [
128+
'stache' => [ // [tl! ++]
129+
'driver' => 'redis', // [tl! ++]
130+
'connection' => 'stache', // [tl! ++]
131+
], // [tl! ++]
132+
],
133+
```
78134

79-
If you plan to use Statamic's [Git Automation](/git-automation) feature alongside zero downtime deployments, you may need to tweak your deployment settings to enable git commits and pushes from each release folder.
135+
Then in `config/statamic/stache.php`, point the Stache at the new store:
80136

81-
### Setting up a Git remote
137+
```php
138+
'cache_store' => 'stache', // [tl! ++]
139+
```
140+
141+
:::tip
142+
The `stache` connection above shares Redis database `1` with the default `cache` connection. Key prefixes keep their entries separate during normal use, but `php artisan cache:clear` wipes both. Point the `stache` connection at a different database (e.g. `'database' => env('REDIS_STACHE_DB', '2')`) for complete isolation, mirroring the file cache option above where the two stores naturally live in separate directories.
143+
:::
144+
145+
## Git Automation
146+
147+
If you plan to use Statamic's [Git Automation](/git-automation) feature alongside zero downtime deployments, you'll need to set up a dedicated git clone for Statamic's content writes so they survive the symlink swap. See [Why this setup is needed](#why-this-setup-is-needed) at the end of this section for the full picture.
82148

83149
:::tip
84-
Unlike other services, Laravel Forge will actually keep the `.git` folder around in each release, meaning you can skip this step.
150+
Examples below use Laravel Forge syntax. The same concepts apply to Envoyer, Ploi, Deployer, or custom scripts; you'll need to adapt the platform-specific syntax.
85151
:::
86152

87-
Most zero downtime deployment services, like [Envoyer](https://envoyer.io/) and [Deployer](https://deployer.org/) create releases _without_ a `.git` folder, which Statamic needs to commit and push content back to your repository.
153+
### Prerequisites
154+
155+
Set a global git identity on the server. Autostash and rebase in [Updating your deploy script](#updating-your-deploy-script) need it:
156+
157+
```bash
158+
git config --global user.name "$(hostname)"
159+
git config --global user.email "$(whoami)@$(hostname).local"
160+
```
161+
162+
### Setting up a Git remote
163+
164+
Create a new `statamic` directory at the root of your site with a [sparse-checkout](https://git-scm.com/docs/git-sparse-checkout) clone inside it. The clone lives outside any release directory, giving Statamic's Git Automation a dedicated working tree where content writes can be committed and pushed reliably.
88165

89-
You can work around this by setting up a Git object right after the `Clone New Release` step of your deployment process:
166+
The sparse-checkout list below covers the paths Statamic tracks by default. We'll wire `config/statamic/git.php` to point at this clone in [Configuring git paths](#configuring-git-paths) further down. Be sure to replace `your-site`, `your-org/your-repo`, and `your-branch` with your values:
90167

91168
```bash
169+
cd your-site
170+
mkdir -p statamic && cd $_
92171
git init
93-
git remote add origin git@github.com:your/remote-repository.git
94-
git fetch
95-
git branch --track main origin/main
96-
git reset HEAD
172+
git remote add origin git@github.com:your-org/your-repo.git
173+
git config core.sparseCheckout true
174+
cat > .git/info/sparse-checkout <<'EOF'
175+
content/
176+
users/
177+
resources/addons/
178+
resources/blueprints/
179+
resources/fieldsets/
180+
resources/forms/
181+
resources/users/
182+
resources/preferences.yaml
183+
resources/sites.yaml
184+
public/assets/
185+
EOF
186+
git fetch origin your-branch
187+
git checkout your-branch
97188
```
98189

99-
Be sure to modify the above remote to point to your remote repository, along with the branch you wish to track.
190+
:::tip
191+
Form submissions are handled separately. See [Committing form submissions](#committing-form-submissions).
192+
:::
100193

101-
### Preventing circular deployments
194+
### Adding shared paths
102195

103-
If you plan on enabling automatic deployment when commits are pushed to your repository, you may wish to selectively disable deployments when Statamic pushes commits back to your repository.
196+
In your deployment tool's shared paths configuration, add an entry for each path in the sparse-checkout list from [Setting up a Git remote](#setting-up-a-git-remote) above.
197+
198+
| From | To |
199+
| ------------------------------------- | ---------------------------- |
200+
| `statamic/content` | `content` |
201+
| `statamic/users` | `users` |
202+
| `statamic/resources/addons` | `resources/addons` |
203+
| `statamic/resources/blueprints` | `resources/blueprints` |
204+
| `statamic/resources/fieldsets` | `resources/fieldsets` |
205+
| `statamic/resources/forms` | `resources/forms` |
206+
| `statamic/resources/users` | `resources/users` |
207+
| `statamic/resources/preferences.yaml` | `resources/preferences.yaml` |
208+
| `statamic/resources/sites.yaml` | `resources/sites.yaml` |
209+
| `statamic/public/assets` | `public/assets` |
210+
211+
### Configuring git paths
104212

105-
To do this, you will first need to append `[BOT]` to Statamic's commit messages [as documented here](/git-automation#customizing-commits). Once this is done, you can add a step to your deployment process to cancel the deployment when the commit message contains `[BOT]`.
213+
In `config/statamic/git.php`, wrap every tracked path in an environment variable so the production paths stay configurable through `.env`. The defaults fall back to the standard Statamic locations when no variable is set.
106214

107215
```php
108-
if [[ $FORGE_DEPLOY_MESSAGE =~ "[BOT]" ]]; then
109-
echo "AUTO-COMMITTED ON PRODUCTION. NOTHING TO DEPLOY."
110-
exit 0
111-
fi
216+
'paths' => [
217+
env('STATAMIC_GIT_CONTENT_PATH', base_path('content')),
218+
env('STATAMIC_GIT_USERS_PATH', base_path('users')),
219+
env('STATAMIC_GIT_ADDONS_PATH', resource_path('addons')),
220+
env('STATAMIC_GIT_BLUEPRINTS_PATH', resource_path('blueprints')),
221+
env('STATAMIC_GIT_FIELDSETS_PATH', resource_path('fieldsets')),
222+
env('STATAMIC_GIT_FORMS_PATH', resource_path('forms')),
223+
env('STATAMIC_GIT_RESOURCE_USERS_PATH', resource_path('users')),
224+
env('STATAMIC_GIT_PREFERENCES_PATH', resource_path('preferences.yaml')),
225+
env('STATAMIC_GIT_SITES_PATH', resource_path('sites.yaml')),
226+
env('STATAMIC_GIT_ASSETS_PATH', public_path('assets')),
227+
],
228+
```
229+
230+
Then add the corresponding values to the site's `.env`, pointing each one at its path inside the `statamic` directory you created above. Be sure to replace `forge/your-site` with your site's path:
231+
112232
```
233+
STATAMIC_GIT_CONTENT_PATH=/home/forge/your-site/statamic/content
234+
STATAMIC_GIT_USERS_PATH=/home/forge/your-site/statamic/users
235+
STATAMIC_GIT_ADDONS_PATH=/home/forge/your-site/statamic/resources/addons
236+
STATAMIC_GIT_BLUEPRINTS_PATH=/home/forge/your-site/statamic/resources/blueprints
237+
STATAMIC_GIT_FIELDSETS_PATH=/home/forge/your-site/statamic/resources/fieldsets
238+
STATAMIC_GIT_FORMS_PATH=/home/forge/your-site/statamic/resources/forms
239+
STATAMIC_GIT_RESOURCE_USERS_PATH=/home/forge/your-site/statamic/resources/users
240+
STATAMIC_GIT_PREFERENCES_PATH=/home/forge/your-site/statamic/resources/preferences.yaml
241+
STATAMIC_GIT_SITES_PATH=/home/forge/your-site/statamic/resources/sites.yaml
242+
STATAMIC_GIT_ASSETS_PATH=/home/forge/your-site/statamic/public/assets
243+
```
244+
245+
### Updating your deploy script
246+
247+
The deploy script below is an example for Laravel Forge with the new lines highlighted. Adapt the `$FORGE_*` variables and macros to your platform's equivalents.
248+
249+
```bash
250+
$CREATE_RELEASE()
113251

114-
### Ensuring proper deployment hook order
252+
cd $FORGE_RELEASE_DIRECTORY
115253

116-
When adding these steps to your deployment process, you should be mindful of the order in which they happen. Here's the order we recommend:
254+
$FORGE_COMPOSER install --no-dev --no-interaction --prefer-dist --optimize-autoloader
255+
$FORGE_PHP artisan optimize
256+
$FORGE_PHP artisan storage:link
117257

118-
* Cancel deployments when commit message contains `[BOT]`
119-
* Create release
120-
* Init Git repository & add Git remote (if necessary)
121-
* The rest of your deployment script...
258+
cd $FORGE_SITE_ROOT/statamic # [tl! ++]
259+
git pull --rebase --autostash origin $FORGE_SITE_BRANCH # [tl! ++]
260+
cd $FORGE_RELEASE_DIRECTORY # [tl! ++]
261+
262+
$FORGE_PHP please stache:warm
263+
$FORGE_PHP please search:update --all
264+
265+
npm ci || npm install
266+
npm run build
267+
268+
$ACTIVATE_RELEASE()
269+
270+
$RESTART_QUEUES()
271+
```
272+
273+
The pull runs before `stache:warm` so the Stache is warmed against the final content state. It brings in any content commits pushed from elsewhere, like edits made in a developer's local environment.
274+
275+
[`--autostash`](https://git-scm.com/docs/git-rebase#Documentation/git-rebase.txt---autostash) makes sure any in-flight writes in the Statamic clone, like a Control Panel edit Statamic's queue hasn't committed yet, survive the pull.
122276

123277
:::tip
124-
If you're using [Static Caching](/static-caching), make sure you warm the cache _after_ updating the current release, otherwise you'll be warming the wrong cache.
278+
If you're using [Static Caching](/static-caching), make sure you warm the cache _after_ activating the current release, otherwise you'll be warming the wrong cache.
125279
:::
126280

127281
### Committing form submissions
128282

129-
If you plan on committing form submissions, you will need to store them outside the shared `storage` directory.
283+
If you plan on committing form submissions, you will need to store them outside the shared `storage` directory.
130284

131-
To customize where form submissions are stored, add a `form-submissions` array to your `config/statamic/stache.php` config file:
285+
First, customize where form submissions are stored by adding a `form-submissions` array to your `config/statamic/stache.php`:
132286

133287
```php
134288
'stores' => [
@@ -139,20 +293,64 @@ To customize where form submissions are stored, add a `form-submissions` array t
139293
],
140294
```
141295

142-
After doing this, you will also need to update the tracked path for your submissions in `config/statamic/git.php`:
296+
Then track the new path by adding an env-wrapped entry to `config/statamic/git.php`:
143297

144298
```php
145299
'paths' => [
146-
base_path('content'),
147-
base_path('users'),
148-
resource_path('blueprints'),
149-
resource_path('fieldsets'),
150-
resource_path('forms'),
151-
resource_path('users'),
152-
resource_path('preferences.yaml'),
153-
resource_path('sites.yaml'),
154-
storage_path('forms'), // [tl! focus --]
155-
base_path('forms'), // [tl! focus ++]
156-
public_path('assets'),
300+
env('STATAMIC_GIT_SUBMISSIONS_PATH', base_path('forms')), // [tl! ++]
157301
],
158302
```
303+
304+
And the matching value to the site's `.env`:
305+
306+
```
307+
STATAMIC_GIT_SUBMISSIONS_PATH=/home/forge/your-site/statamic/forms
308+
```
309+
310+
Finally, populate `forms/` in the Statamic clone and add a matching shared path. See steps 1 and 2 of [Adding paths later](#adding-paths-later).
311+
312+
### Adding paths later
313+
314+
To track and commit additional paths after the initial setup, follow the same order as the main flow:
315+
316+
1. **[Populate the path in the Statamic clone](#setting-up-a-git-remote):**
317+
318+
```bash
319+
cd /home/forge/your-site/statamic
320+
git sparse-checkout add <path>/
321+
```
322+
323+
2. **[Add a new shared path](#adding-shared-paths)**.
324+
325+
3. **[Configure the git path](#configuring-git-paths)**.
326+
327+
### Preventing circular deployments
328+
329+
If you plan on enabling automatic deployment when commits are pushed to your repository, you may wish to selectively disable deployments when Statamic pushes commits back to your repository.
330+
331+
To do this, you will first need to append `[BOT]` to Statamic's commit messages [as documented here](/git-automation#customizing-commits). Once this is done, add the following at the very top of your deploy script, before the release is created, so the check happens before any deployment work is done:
332+
333+
```bash
334+
if [[ $FORGE_DEPLOY_MESSAGE =~ "[BOT]" ]]; then
335+
echo "AUTO-COMMITTED ON PRODUCTION. NOTHING TO DEPLOY."
336+
exit 0
337+
fi
338+
```
339+
340+
### Why this setup is needed
341+
342+
Statamic's Git Automation tracks paths defined in [config/statamic/git.php](#configuring-git-paths) using helpers like `base_path('content')` and `resource_path('forms')`, which resolve to whichever release the running process is in. Without intervention, this creates two failure modes during zero-downtime deploys.
343+
344+
#### Silent skipped commits
345+
346+
A Control Panel edit during a deploy lands in the old release's content directory. After the symlink swap and queue restart, the queued commit job runs from the _new_ release, reads its path config against the new release's `base_path()`, finds a clean working tree, and exits without committing. No error is raised. When the old release is later cleaned up, the write is gone.
347+
348+
#### Push rejections
349+
350+
Each release ships with its own `.git/`. If something else pushes to origin (a developer pushing code, a second server, an overlapping worker from a previous release), the release's local history can fall behind origin and the push gets rejected:
351+
352+
```
353+
error: failed to push some refs to ''. Updates were rejected because the remote contains work that you do not have locally.
354+
```
355+
356+
[Shared paths](#adding-shared-paths) fix the first by routing writes to a persistent location outside any release, so Statamic's commits always see them. The dedicated [Statamic clone](#setting-up-a-git-remote) fixes the second by giving Statamic a single `.git/` that persists across deploys instead of being a fresh clone every time. The [deploy script's `git pull`](#updating-your-deploy-script) resyncs it with any external commits at deploy time.

0 commit comments

Comments
 (0)