MongoDB-k8s fails to start, stuck in a crash loop of exit status 48.

[amc94]

Steps to reproduce

1. Deploy MongoDB-K8s on Canonicalk8s 1.32

Expected behavior

MongoDB comes up

Actual behavior

MongoDB is stuck starting with the status message of Waiting for replica set initialisation

Versions

100

Operating system:

Juju CLI:

Juju agent: 3.6.9

Charm revision: 100

microk8s:

Log output

Juju debug log:

-----
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-target-0/charm/venv/lib/python3.12/site-packages/single_kernel_mongo/core/k8s_workload.py", line 68, in restart
    self.container.restart(self.service)
  File "/var/lib/juju/agents/unit-target-0/charm/venv/lib/python3.12/site-packages/ops/model.py", line 2451, in restart
    self._pebble.restart_services(service_names)
  File "/var/lib/juju/agents/unit-target-0/charm/venv/lib/python3.12/site-packages/ops/pebble.py", line 2374, in restart_services
    return self._services_action('restart', services, timeout, delay)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/lib/juju/agents/unit-target-0/charm/venv/lib/python3.12/site-packages/ops/pebble.py", line 2401, in _services_action
    raise ChangeError(change.err, change)
ops.pebble.ChangeError: cannot perform the following tasks:
- Start service "mongod" (service start attempt: exited quickly with code 48, will restart)
----- Logs from task 1 -----
2025-11-27T11:35:27Z INFO Most recent service output:
    (...)
    {"t":{"$date":"2025-11-27T11:35:19.251Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:19.251Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:20.106Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:20.107Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:22.378Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:22.378Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:22.568Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:22.568Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:23.050Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:23.050Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:23.183Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:23.183Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:23.401Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:23.402Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:24.201Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:24.202Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:25.354Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:25.354Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
    {"t":{"$date":"2025-11-27T11:35:27.774Z"},"s":"W",  "c":"CONTROL",  "id":29117,   "ctx":"main","msg":"The storage.journal.enabled option and the corresponding --journal and --nojournal command-line options have no effect in this version of Percona Server for MongoDB. Journaling is always enabled. Please remove those options from the config","tags":["startupWarnings"]}
    {"t":{"$date":"2025-11-27T11:35:27.775Z"},"s":"I",  "c":"CONTROL",  "id":5760901, "ctx":"main","msg":"Applied --setParameter options","attr":{"serverParameters":{"processUmask":{"default":63,"value":31}}}}
2025-11-27T11:35:27Z ERROR service start attempt: exited quickly with code 48, will restart
-----
unit-target-0: 2025-11-27 11:35:29 DEBUG juju.apiserver.http api <- fd:-1 ServeHTTP GET /liveness

Additional context

Test Observer Logs https://test-observer.canonical.com/#/charms/290626?testExecutionId=290290&testResultId=8012877 (canonical vpn required)

[syncronize-issues-to-jira]

Thank you for reporting your feedback to us!

The internal ticket has been created: https://warthogs.atlassian.net/browse/DPE-9042.

This message was autogenerated

[patriciareinoso]

Hi @amc94 thank you for reporting this. We will take a look after the end of the year break.

[patriciareinoso]

Revision 100 is on 8/edge branch (https://charmhub.io/mongodb-k8s?channel=8/edge)

[Gu1nness]

MongoDB exit code 48 means that there was an error preventing MongoDB from listening for incoming connections.
Usually, this means the port we use (27017) is already in use, however this might not be the case here as you're using Kubernetes.
You are using Canonical K8s, have you enabled the dns and ingress plugins ? That could be the issue.
Otherwise, I would suspect a network layer issue, depending on how your connection with the host machine is configured.

Otherwise, can you provide us the logs that you can find in /var/log/mongod/mongod.log so that we can investigate a bit further?

[amc94]

Hi @Gu1nness

We're deploying on the default config of canonical k8s, so dns enabled but not ingress. I see we have tests that passed in the same environment however we don't collect the /var/log/mongod/mongod.log