DEPLOY FIEF ON GCP #336

MathieuGrosso · 2024-02-14T15:58:41Z

MathieuGrosso
Feb 14, 2024

[supervisord]
nodaemon=true
logfile=/tmp/supervisord.log
pidfile=/tmp/supervisord.pid
childlogdir=/tmp

[program:fief-server]
command=/bin/zsh -c "fief run-server --port 8000"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:fief-worker]
command=/bin/zsh -c "fief run-worker -p 1 -t 1"
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

[program:redis]
command=/usr/local/bin/redis-server --save 60 1 --loglevel warning
autostart=true
autorestart=true
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0

hello i am trying to deploy fief directly on gcp vm since my db is on a VPC. I need to deploy fief in the vpc so in one of our gcp vm.
I am struggling to deploy redis on the vm with supervisord. Indeed, compute engine does not accept docker compose so i need to use supervisord to make it work
If by any chance someone knows how to do this let me know.

---- French version !

Bonjour !
On essaie de déployer fief, on voulait essayer ce package pour gérer l'auth mais on a du mal à le déployer sur gcp avec supervisord. Le principal suspect étant redis. Pour l'instant voilà l'erreur:

> 2024-02-14 16:48:11,887 INFO supervisord started with pid 6087
> 2024-02-14 16:48:12,892 INFO spawned: 'fief-server' with pid 6092
> 2024-02-14 16:48:12,895 INFO spawned: 'fief-worker' with pid 6093
> 2024-02-14 16:48:13,897 INFO success: fief-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
> 2024-02-14 16:48:13,897 INFO success: fief-worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
> INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
> INFO  [alembic.runtime.migration] Will assume transactional DDL.
> [2024-02-14 16:48:19,694] [PID 6093] [MainThread] [dramatiq.MainProcess] [INFO] Dramatiq '1.14.2' is booting up.
> 2024-02-14 16:48:19.452 | INFO     | fief.worker:<module>:19 - Fief Worker started - {"version": "0.27.0"}
> 2024-02-14 16:48:19.684 | INFO     | dramatiq.cli:worker_process:412 - Worker process is ready for action. - {}
> 2024-02-14 16:48:19.736 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:19.737 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:19.738 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:19.739 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> [2024-02-14 16:48:20,214] [PID 6105] [MainThread] [dramatiq.ForkProcess(1)] [INFO] Fork process 'dramatiq.middleware.prometheus:_run_exposition_server' is ready for action.
> Main Fief workspace already exists
> [2024-02-14 16:48:21,160] [PID 6104] [MainThread] [dramatiq.ForkProcess(0)] [INFO] Fork process 'fief.scheduler:schedule' is ready for action.
> [2024-02-14 16:48:21,166] [PID 6104] [MainThread] [apscheduler.scheduler] [INFO] Adding job tentatively -- it will be properly scheduled when the scheduler starts
> [2024-02-14 16:48:21,166] [PID 6104] [MainThread] [apscheduler.scheduler] [INFO] Adding job tentatively -- it will be properly scheduled when the scheduler starts
> [2024-02-14 16:48:21,166] [PID 6104] [MainThread] [apscheduler.scheduler] [INFO] Added job "Actor.send" to job store "default"
> [2024-02-14 16:48:21,166] [PID 6104] [MainThread] [apscheduler.scheduler] [INFO] Added job "Actor.send" to job store "default"
> [2024-02-14 16:48:21,166] [PID 6104] [MainThread] [apscheduler.scheduler] [INFO] Scheduler started
> Main Fief user already exists
> Main Fief admin API key not provided in settings. Skipping its creation.
> 2024-02-14 16:48:22.740 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:22.741 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:22.743 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:22.743 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> INFO:     Started server process [6092]
> INFO:     Waiting for application startup.
> 2024-02-14 16:48:24.443 | INFO     | fief.lifespan:lifespan:28 - Fief Server started - {"version": "0.27.0"}
> 2024-02-14 16:48:24.443 | WARNING  | fief.lifespan:lifespan:31 - Telemetry is enabled.
> We will collect data to better understand how Fief is used and improve the project.
> You can opt-out by setting the environment variable `TELEMETRY_ENABLED=false`.
> Read more about Fief's telemetry here: https://docs.fief.dev/telemetry - {}
> 2024-02-14 16:48:25.751 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:25.751 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:25.755 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:25.756 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:28.759 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:28.761 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:28.765 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:28.766 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:31.769 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:31.770 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:31.772 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}
> 2024-02-14 16:48:31.773 | INFO     | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {}
> 2024-02-14 16:48:34.779 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}

Si quelqu'un a une idée, faites le nous savoir !

Bonne journée.

Answered by frankie567

Feb 20, 2024

@louistransfer Nothing wrong with what you show me here. When accessing the root path / without a valid user session, Fief returns a redirection to /login (hence the 307 status code). By default, cURL doesn't follow redirections. You can add the -L flag to tell it to automatically follow redirections: https://everything.curl.dev/http/redirects#tell-curl-to-follow-redirects

@MathieuGrosso Yes, normally that's something proxy are doing by default, this is quite standard. GCP docs even claim it's the case: https://cloud.google.com/load-balancing/docs/https

...

I just had another look at your Dockerfile. I see that you are copying a .env file. Are you setting FORWARDED_ALLOW_IPS inside that f…

View full answer

@MathieuGrosso · 2024-02-14T15:58:45Z

fief-bailiff[bot]
bot Feb 14, 2024

Hail, @MathieuGrosso 👋 Welcome to Fief's kingdom!

Our team will get back to you very soon to help.

In the meantime, take a minute to star our repository ⭐️

Farewell!

0 replies

frankie567 · 2024-02-14T16:40:40Z

frankie567
Feb 14, 2024
Maintainer

Salut @MathieuGrosso 👋

If I understand correctly, you're running Redis on the same machine/VM than Fief itself, processes being managed by supervisord. From the logs, it seems that Fief tries to reach Redis using the hostname redis:

Error 8 connecting to redis:6379. nodename nor servname provided, or not known. - {}

If Redis does indeed run on the same machine/VM, then you probably should reach it directly on localhost. So, I suggest you to try the following environment variable:

REDIS_URL=redis://localhost:6379

(which is actually the default if you don't set it)

0 replies

MathieuGrosso · 2024-02-14T16:57:17Z

MathieuGrosso
Feb 14, 2024
Author

Salut François 🙋🙋🙋! I just tried what you said, it seems to work locally but not on a compute engine vm. I get this error: ``` 2024-02-14 17:04:18,736 CRIT Supervisor is running as root. Privileges were not dropped because no user is specified in the config file. If you intend to run as root, you can set user=root in the config file to avoid this message. 2024-02-14 17:04:18,737 INFO supervisord started with pid 1 2024-02-14 17:04:19,739 INFO spawned: 'fief-server' with pid 7 2024-02-14 17:04:19,742 INFO spawned: 'fief-worker' with pid 8 2024-02-14 17:04:19,742 INFO spawnerr: can't find command '/usr/local/bin/redis-server' 2024-02-14 17:04:20,743 INFO success: fief-server entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2024-02-14 17:04:20,744 INFO success: fief-worker entered RUNNING state, process has stayed up for > than 1 seconds (startsecs) 2024-02-14 17:04:20,744 INFO spawnerr: can't find command '/usr/local/bin/redis-server' 2024-02-14 17:04:22,745 INFO spawnerr: can't find command '/usr/local/bin/redis-server' INFO [alembic.runtime.migration] Context impl PostgresqlImpl. INFO [alembic.runtime.migration] Will assume transactional DDL. 2024-02-14 17:04:25,894 INFO spawnerr: can't find command '/usr/local/bin/redis-server' 2024-02-14 17:04:25,895 INFO gave up: redis entered FATAL state, too many start retries too quickly [2024-02-14 17:04:26,397] [PID 8] [MainThread] [dramatiq.MainProcess] [INFO] Dramatiq '1.14.2' is booting up. 2024-02-14 17:04:26.259 | INFO | fief.worker:<module>:19 - Fief Worker started - {"version": "0.27.0"} 2024-02-14 17:04:26.390 | INFO | dramatiq.cli:worker_process:412 - Worker process is ready for action. - {} [2024-02-14 17:04:26,400] [PID 16] [MainThread] [dramatiq.ForkProcess(1)] [INFO] Fork process 'dramatiq.middleware.prometheus:_run_exposition_server' is ready for action. 2024-02-14 17:04:26.409 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:26.409 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} 2024-02-14 17:04:26.410 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:26.410 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} Main Fief workspace already exists Main Fief user already exists Main Fief admin API key not provided in settings. Skipping its creation. [2024-02-14 17:04:27,974] [PID 15] [MainThread] [dramatiq.ForkProcess(0)] [INFO] Fork process 'fief.scheduler:schedule' is ready for action. [2024-02-14 17:04:27,978] [PID 15] [MainThread] [apscheduler.scheduler] [INFO] Adding job tentatively -- it will be properly scheduled when the scheduler starts [2024-02-14 17:04:27,978] [PID 15] [MainThread] [apscheduler.scheduler] [INFO] Adding job tentatively -- it will be properly scheduled when the scheduler starts [2024-02-14 17:04:27,979] [PID 15] [MainThread] [apscheduler.scheduler] [INFO] Added job "Actor.send" to job store "default" [2024-02-14 17:04:27,980] [PID 15] [MainThread] [apscheduler.scheduler] [INFO] Added job "Actor.send" to job store "default" [2024-02-14 17:04:27,980] [PID 15] [MainThread] [apscheduler.scheduler] [INFO] Scheduler started 2024-02-14 17:04:29.411 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:29.412 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} 2024-02-14 17:04:29.413 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:29.413 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} INFO: Started server process [7] INFO: Waiting for application startup. 2024-02-14 17:04:32.413 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:32.413 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} 2024-02-14 17:04:32.414 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:32.414 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} 2024-02-14 17:04:32.569 | INFO | fief.lifespan:lifespan:28 - Fief Server started - {"version": "0.27.0"} 2024-02-14 17:04:32.569 | WARNING | fief.lifespan:lifespan:31 - Telemetry is enabled. We will collect data to better understand how Fief is used and improve the project. You can opt-out by setting the environment variable `TELEMETRY_ENABLED=false`. Read more about Fief's telemetry here: https://docs.fief.dev/telemetry - {} 2024-02-14 17:04:35.414 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:35.414 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} 2024-02-14 17:04:35.415 | CRITICAL | dramatiq.worker:run:276 - Consumer encountered a connection error: Error 111 connecting to localhost:6379. Connection refused. - {} 2024-02-14 17:04:35.415 | INFO | dramatiq.worker:run:288 - Restarting consumer in 3.00 seconds. - {} ```

23 replies

MathieuGrosso Feb 19, 2024
Author

I have been digging it, for now i dont see anything about X-forwarded-proto header.
Possibly we could define the "En-têtes de requêtes personnalisés" or the "En-têtes de réponses personnalisé" to X-Forwaded-Proto:https but i am not sure if this it.
i will update this when I have found it :)

Thanks !

MathieuGrosso Feb 19, 2024
Author

Technically, GCP automatically adds the X-forwarded-proto header to requests so it should work, but it does not appear to be the case

louistransfer Feb 19, 2024

Hello !
I am also working on this issue with Mathieu and I am trying to access the user page, I believe that the redirection is working but Fief doesn't register the redirection :

curl -v https://fief.xxx/
Trying 31.xx.xxx.xxx:443...
Connected to fief.xxx (31.xx.xxx.xxx) port 443 (#0)
ALPN: offers h2,http/1.1
(304) (OUT), TLS handshake, Client hello (1):
CAfile: /etc/ssl/cert.pem
CApath: none
(304) (IN), TLS handshake, Server hello (2):
(304) (IN), TLS handshake, Unknown (8):
(304) (IN), TLS handshake, Certificate (11):
(304) (IN), TLS handshake, CERT verify (15):
(304) (IN), TLS handshake, Finished (20):
(304) (OUT), TLS handshake, Finished (20):
SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
ALPN: server accepted h2
Server certificate:
subject: CN=fief.xxx
start date: Feb 16 14:46:43 2024 GMT
expire date: May 16 15:39:37 2024 GMT
subjectAltName: host "fief.xx" matched cert's "fief.xxx"
issuer: C=US; O=Google Trust Services LLC; CN=GTS CA 1D4
SSL certificate verify ok.
using HTTP/2
h2 [:method: GET]
h2 [:scheme: https]
h2 [:authority: fief.xxx]
h2 [:path: /]
h2 [user-agent: curl/8.1.2]
h2 [accept: /]
Using Stream ID: 1 (easy handle 0x15500c600)
GET / HTTP/2
Host: fief.xxx
User-Agent: curl/8.1.2
Accept: /
< HTTP/2 307
< date: Mon, 19 Feb 2024 17:32:12 GMT
< server: uvicorn
< location: http://fief.xxx/login
< content-length: 31
< content-type: application/json
< content-security-policy: frame-ancestors 'self'
< x-frame-options: SAMEORIGIN
< referrer-policy: strict-origin-when-cross-origin
< x-content-type-options: nosniff
< permissions-policy: geolocation=(), camera=(), microphone=()
< via: 1.1 google
< x-forwarded-proto: https
< alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
<
Connection #0 to host fief.xxx left intact
{"detail":"Temporary Redirect"}%

frankie567 Feb 20, 2024
Maintainer

@louistransfer Nothing wrong with what you show me here. When accessing the root path / without a valid user session, Fief returns a redirection to /login (hence the 307 status code). By default, cURL doesn't follow redirections. You can add the -L flag to tell it to automatically follow redirections: https://everything.curl.dev/http/redirects#tell-curl-to-follow-redirects

@MathieuGrosso Yes, normally that's something proxy are doing by default, this is quite standard. GCP docs even claim it's the case: https://cloud.google.com/load-balancing/docs/https

...

I just had another look at your Dockerfile. I see that you are copying a .env file. Are you setting FORWARDED_ALLOW_IPS inside that file? If yes, then it won't work 😅

The fact that Fief is able to parse .env file is because we use a specific wrapper for our settings, which is able to fallback to the values in this file if they are not available in OS environment variables. However, FORWARDED_ALLOW_IPS is not a Fief setting; it's something that's expected by the underlying Python server, Uvicorn. So it must be a true OS environment variable (like you set GCP_PROJECT at the beginning of your Dockerfile).

Answer selected by MathieuGrosso

MathieuGrosso Feb 20, 2024
Author

yes it was exactly it, i have set it directly in the supervisord conf to pass it as env variable to the python server and it made it work. I saw it when printing env variable of the process

Thanks a lto

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fief

DEPLOY FIEF ON GCP #336

{{title}}

Replies: 3 comments 23 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

Select a reply

Fief

DEPLOY FIEF ON GCP #336

MathieuGrosso Feb 14, 2024

Replies: 3 comments · 23 replies

fief-bailiff[bot] bot Feb 14, 2024

frankie567 Feb 14, 2024 Maintainer

MathieuGrosso Feb 14, 2024 Author

MathieuGrosso Feb 19, 2024 Author

MathieuGrosso Feb 19, 2024 Author

louistransfer Feb 19, 2024

frankie567 Feb 20, 2024 Maintainer

MathieuGrosso Feb 20, 2024 Author

MathieuGrosso
Feb 14, 2024

Replies: 3 comments 23 replies

fief-bailiff[bot]
bot Feb 14, 2024

frankie567
Feb 14, 2024
Maintainer

MathieuGrosso
Feb 14, 2024
Author

MathieuGrosso Feb 19, 2024
Author

MathieuGrosso Feb 19, 2024
Author

frankie567 Feb 20, 2024
Maintainer

MathieuGrosso Feb 20, 2024
Author