[Bug]: Container coolify is unhealthy

[nullndr]

Error Message and Logs

I'm trying to install coolify into a fresh server from hetzenr and it is all good until the point 9 that exit without notify:

   _____            _ _  __
  / ____|          | (_)/ _|
 | |     ___   ___ | |_| |_ _   _
 | |    / _ \ / _ \| | |  _| | | |
 | |___| (_) | (_) | | | | | |_| |
  \_____\___/ \___/|_|_|_|  \__, |
                             __/ |
                            |___/

Welcome to Coolify Installer!
This script will install everything for you. Sit back and relax.
Source code: https://github.com/coollabsio/coolify/blob/main/scripts/install.sh

---------------------------------------------
| Operating System  | arch rolling
| Docker            | 26.0
| Coolify           | 4.0.0-beta.341
| Helper            | 1.0.1
| Realtime          | 1.0.1
---------------------------------------------

1. Installing required packages (curl, wget, git, jq).
warning: curl-8.10.1-1 is up to date -- skipping
warning: wget-1.24.5-3 is up to date -- skipping
warning: git-2.46.1-1 is up to date -- skipping
warning: jq-1.7.1-2 is up to date -- skipping
2. Check OpenSSH server configuration.
 - OpenSSH server is installed.
 - SSH PermitRootLogin is enabled.
3. Check Docker Installation.
 - Docker is installed.
4. Check Docker Configuration.
 - Docker configuration is up to date.
5. Download required files from CDN.
6. Make backup of .env to .env-20240918-171420
7. Propagating .env with new values - if necessary.
8. Checking for SSH key for localhost access.
9. Installing Coolify (4.0.0-beta.340)
 - It could take a while based on your server's performance, network speed, stars, etc.
 - Please wait.
 - Until then, here's a joke for you:

Two SQL tables sit at the bar. A query approaches and asks "Can I join you?"

[root@archlinux ~]#

I tried then to run the upgrade.sh script with the outputs with the following result:

[root@archlinux ~]# bash /data/coolify/source/upgrade.sh "${LATEST_VERSION:-latest}" "${LATEST_HELPER_VERSION:-latest}"
 Container coolify-realtime  Creating
 Container coolify-redis  Creating
 Container coolify-db  Creating
 Container coolify-realtime  Created
 Container coolify-db  Created
 Container coolify-redis  Created
 Container coolify  Creating
 Container coolify  Created
 Container coolify-realtime  Starting
 Container coolify-db  Starting
 Container coolify-redis  Starting
 Container coolify-db  Started
 Container coolify-redis  Started
 Container coolify-realtime  Started
 Container coolify-db  Waiting
 Container coolify-redis  Waiting
 Container coolify-realtime  Waiting
 Container coolify-redis  Healthy
 Container coolify-realtime  Healthy
 Container coolify-db  Healthy
 Container coolify  Starting
 Container coolify  Started
 Container coolify  Waiting
 Container coolify-db  Waiting
 Container coolify-redis  Waiting
 Container coolify-realtime  Waiting
 Container coolify-redis  Healthy
 Container coolify-db  Healthy
 Container coolify-realtime  Healthy
container coolify is unhealthy

Coolify Version

v4.0.0-beta.341

Are you using Coolify Cloud?

No (self-hosted)

Operating System and Version (self-hosted)

Arch Linux

[nullndr]

Looks like this is a problem with Arch Linux, I just tried to install the same version on a fresh debian server and it installed with no issues.

Do you have any idea about why the installation on Arch Linux is not working?

[leoboyerbx]

I do have this problem on a fresh ubuntu server, so it's not an Arch-only issue

[andrewseif]

I do have this problem on a fresh ubuntu server, so it's not an Arch-only issue

What version of ubuntu?

[sandroschwarz]

Hey everyone, having the same problem during the upgrade process from v4.0.0-beta.341 to v4.0.0-beta.345 under a self-hosted version of Ubuntu 24.04.1.
Checking the output of docker logs coolify-redis yields the following message over and over:

*** FATAL CONFIG FILE ERROR (Redis 7.4.0) ***
Reading the configuration file, at line 4                                                                                                                                                                                              >>> 'requirepass'
wrong number of arguments

Reading through the install script for Coolify I gathered there's a prod.yml being spun up.
Checking /data/coolify/source/docker-compose.prod.yml didn't show any oddities, so I checked the env file under /data/coolify/source/.env, which had its contents for keys APP_ID, APP_KEY, DB_PASSWORD and REDIS_PASSWORD set to nothing/blank.

Simply manually re-setting them with freshly set variables didn't help much, trying to install the update now yields a "healthy" redis service, but instead "container coolify is unhealthy". As it seems because Coolify now can't connect to the database anymore because of the missing/wrong credentials.

Anything else one could try?

EDIT: I was not able to recover from this on my own and decided to uninstall Coolify via the official documentation, performed a docker system prune and reinstalled Coolify. I have now implemented a cron job to back up the contents of /data/coolify/source/.env, as recommended by the update script, in hopes I can recover from that in the future.

[danielalves96]

I have the same error on other vps. 😞

[halowahyudi]

same this error on Ubuntu 20.04 lts

[bocklighting]

was anyone able to resolve this issue? having the same issue. tried reinstalling coolify, no luck. also using ubutnu 24.04 lts

coolify reddis container logs

*** FATAL CONFIG FILE ERROR (Redis 7.4.0) ***
Reading the configuration file, at line 4

'requirepass'
wrong number of arguments

[truekasun]

I got the same issue once I tried to upgrade Coolify.

Since the Coolify container was failing I checked the log of that container via docker logs --since=1h 'container_id_here' and this was the log.

/ ___|  ___ _ ____   _____ _ __  / ___|(_) __| | ___  | | | |_ __
\___ \ / _ \  __\ \ / / _ \  __| \___ \| |/ _` |/ _ \ | | | |  _ \
___) |  __/ |   \ V /  __/ |     ___) | | (_| |  __/ | |_| | |_) |
|____/ \___|_|    \_/ \___|_|    |____/|_|\__,_|\___|  \___/| .__/
                                                            |_|

Brought to you by serversideup.net
--------------------------------------------------------------------

To support Server Side Up projects visit:
https://serversideup.net/sponsor
-------------------------------------
GID/UID
-------------------------------------

User uid:    9999
User gid:    9999
-------------------------------------

🔓 SSL_MODE has been DISABLED, setting the web server to work in HTTP only...
🏃‍♂️ Checking for Laravel automations...
🔐 Linking the storage...

   ERROR  The [public/storage] link already exists.


   INFO  Nothing to migrate.

[12-Oct-2024 11:58:15] NOTICE: fpm is running, pid 99
[12-Oct-2024 11:58:15] NOTICE: ready to handle connections
[12-Oct-2024 11:58:15] NOTICE: systemd monitor interval set to 10000ms

   INFO  Seeding database.

Running in self-hosted mode.

In Connection.php line 820:

  SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates u
  nique constraint "servers_pkey"
  DETAIL:  Key (id)=(0) already exists. (Connection: pgsql, SQL: insert into
  "servers" 

  ("id", "name", "description", "user", "ip", "team_id", "private_key_id", "proxy", "uuid", "updated_at", "created_at")
  values (0, localhost,This is the server where Coolify is running on. Don't delete this!, root, host.docker.internal, 0, 0, {"type":"TRAEFIK","status":"exited"}, bswk0k0okwckwco08o4oo0sk, 2024-10-12 11:58:15, 2024-10-12 11:58:15) returning "id")


In Connection.php line 412:

  SQLSTATE[23505]: Unique violation: 7 ERROR:  duplicate key value violates unique constraint "servers_pkey"
  DETAIL:  Key (id)=(0) already exists.


s6-rc: warning: unable to start service init-seeder: command exited 1
/run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
prog: fatal: stopping the container.
[12-Oct-2024 11:58:16] NOTICE: Terminating ...
[12-Oct-2024 11:58:16] NOTICE: exiting, bye-bye!

This could be because I have clicked twice on the Upgrade button on the dashboard alert.

All the services are working fine except for the Coolify container.

---

UPDATE 1

I was able to resolve this (not the best way to do it).

1. I logged into the database (coolify-db) docker container (via docker exec -it container_id_here sh)
2. Logged into PostgreSQL (via psql -U coolify)
3. Selected the database (via \c coolify)
4. Checked the database rows on the servers table (via select * from "servers";)
5. There were no records. Then I tried to execute the insert statement from the above log. Then I could reproduce the same error in the log. (duplicate key value violates unique constraint "servers_pkey" DETAIL: Key (id)=(0) already exists.
6. Since there's no way to disable a unique constrain (or there is, I'm not much familiar with Postgres), I had to delete this constrain
7. When I tried to delete the constrain, it said there are other objects related to this on the additional_destinations table
8. Then checked the additional_destinations table and it was empty as the servers table
9. Then I deleted the servers_pkey unique constrain (via ALTER TABLE servers drop CONSTRAINT servers_pkey cascade;)
10. Once this is done, I restarted the coolify docker container (via docker restart container_id_here)

As of now, the Coolify dashboard is working fine, but I'm not sure what will happen when I try to upgrade again (as migrations will be running again). If there's anything, I'll update it here.

I totally understand this is not the right way to resolve it. But posting what I have observed here assuming it may help someone reproducing this issue or to someone who's having the same issue as me but anyhow need to get it online.

[italianconcerto]

Same for me. I'm running a debian 12 container on proxmox.