Docker containers keep restarting

[radtrentasei]

I have done a fresh install of netpalm but the containers do not stay UP.

(py3venv) [developer@devbox ~]$ docker ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED             STATUS                          PORTS                                       NAMES
802b12dd5126   netpalm_netpalm-worker-fifo     "python3 worker.py f…"   About an hour ago   Restarting (1) 26 seconds ago                                               netpalm_netpalm-worker-fifo_1
ce39b3e672e8   netpalm_netpalm-worker-pinned   "python3 worker.py p…"   About an hour ago   Restarting (1) 6 seconds ago                                                netpalm_netpalm-worker-pinned_1
706c38ed13ec   netpalm_netpalm-controller      "/bin/sh -c 'gunicor…"   About an hour ago   Up 7 seconds                    0.0.0.0:9000->9000/tcp, :::9000->9000/tcp   netpalm_netpalm-controller_1
300a7e0bfa37   netpalm_redis                   "docker-entrypoint.s…"   About an hour ago   Up About an hour                6379/tcp                                    netpalm_redis_1

This is the error I am getting when running docker-compose not in background

netpalm-worker-fifo_1    | TypeError: To define root models, use `pydantic.RootModel` rather than a field called '__root__'

[empusas]

The problem is the pedantic version

In the requirement.txt most python module have no version or pinned down to a specific version:

requirements.txt
fastapi
ttp
netmiko==3.3.2
napalm
ncclient==0.6.9
requests
redis==4.5.1
rq
xmltodict
jinja2
jinja2schema
jsonschema
genie
pyyaml
cachelib==0.3.0
python-redis-lock
filelock
jsonpath_ng
apscheduler==3.6.3
puresnmp==1.9.1
pydantic
names_generator==0.1.0

That is bad practice and exactly causes such issues. The proper way to define requirement is to define a version range.
Here pedantic release a new major version 2.0 on 2023-06-30, the last version 1, v1.10.13 to be precise was released on 2023-09-27. The code was not adjusted to comply with the latest version and as no version was given in the requirements.txt the latest version is used and causes this issue.

if you change the line in the requirements.txt for the pydantic module, rebuilt the containers it works again.
pydantic>=1.10.13,<2.0

Usually all dependencies in requirements.txt should have a range from current to next major/minor release to get all security updates, but avoid problems with breaking changes between versions.

Pining down modules to specific version is also a bad idea as you miss out on all security updates.

[radtrentasei]

Thanks, any chance we can integrate in the main repo?
I am happy to trigger the PR if needed.

[tbotnz]

Feel free to open PR and i will look to merge

[radtrentasei]

I've stopped the PL because I've noticed that the containers are still reloading. This time I think the redis certificate has expired:

netpalm-controller_1     | redis.exceptions.ConnectionError: Error 1 connecting to redis:6379. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1131).

@tbotnz would you be able to help?

[empusas]

There is a script in the main directory of the repo called "redis_gen_new_certs.sh". Run the script, then rebuild the containers. It will work after that.
You can then include the new certificates in your PR.

[radtrentasei]

I've been able to make them work and I've triggered the PR. Please review

[radtrentasei]

@tbotnz any chance you can review please?

[californinson]

I'm having the same issue:

labvm@labvm:~/netpalm$ sudo docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES fb1b6fd4a9fd netpalm_netpalm-worker-pinned "python3 worker.py p…" 48 seconds ago Restarting (1) 4 seconds ago netpalm_netpalm-worker-pinned_1 e26fd59d7df7 netpalm_netpalm-worker-fifo "python3 worker.py f…" 48 seconds ago Restarting (1) 4 seconds ago netpalm_netpalm-worker-fifo_1 ebea78f69ff8 netpalm_netpalm-controller "/bin/sh -c 'gunicor…" 48 seconds ago Restarting (3) 4 seconds ago netpalm_netpalm-controller_1 f448d8b87443 netpalm_redis "docker-entrypoint.s…" 50 seconds ago Up 47 seconds 6379/tcp netpalm_redis_1

I could see that the requirements.txt was fixed after this issue was initially raised, but I'm experiencing the same behaviour.