Process gets stuck and stops responding to requests

[coldsobolev]

Hello, I am using your image as DNS and DHCP server in an office environment and I run it with network_mode: host. For some reason, the container gets stuck and even restarting it takes a very long time. docker logs show nothing useful and even running it in debug mode is not useful. Any idea what could be causing this?

[danil-smirnov]

I have the same issue

[kda-jt]

Agreed, same problem here as well

[kroese]

I dont have a clue how it can be caused. I just launch the dnsmasq executable, so Im not sure what I can do differently. If anyone has an idea, Im all ears.

[kda-jt]

I've set a healthcheck and a restart policy to my docker compose file, this way autoheal takes care of restarting the container if it fails.

    restart: unless-stopped
    healthcheck:
      test: ["CMD", "nslookup", "home.lan", "192.168.1.60"]
      interval: 30s
      timeout: 10s
      retries: 15
      start_period: 15s

I haven't had any issues in a while, the container has not restarted since I last rebooted the host machine (I've checked the uptime on both). Will report back if I see any problems again

UPDATE: I managed to see this behaviour once again! The dnsmasq server stopped responding, so I dropped into a shell inside the container and ran top, the process what still running. I restarted the container and the issue was fixed, but that was really odd.

[maxduke]

same here

I've set a healthcheck and a restart policy to my docker compose file, this way autoheal takes care of restarting the container if it fails.

    restart: unless-stopped
    healthcheck:
      test: ["CMD", "nslookup", "home.lan", "192.168.1.60"]
      interval: 30s
      timeout: 10s
      retries: 15
      start_period: 15s

I haven't had any issues in a while, the container has not restarted since I last rebooted the host machine (I've checked the uptime on both). Will report back if I see any problems again

UPDATE: I managed to see this behaviour once again! The dnsmasq server stopped responding, so I dropped into a shell inside the container and ran top, the process what still running. I restarted the container and the issue was fixed, but that was really odd.

[Feriman22]

How often does this happen to you? I'm trying to figure out what the problem might be, as a hobby docker guy on the internet.

Edit: I have created an advanced healthcheck:

#!/bin/sh
# Multi-level healthcheck for dnsmasq with timeouts

DNS_SERVER="192.168.1.30"
TIMEOUT=3

# 1. Try local domain (test local config) with timeout
if timeout $TIMEOUT nslookup mylocaldomain.duckdns.org "$DNS_SERVER" >/dev/null 2>&1; then
    exit 0
fi

# 2. If that failed, try port availability (is DNS service running?)
if timeout $TIMEOUT nc -zvu "$DNS_SERVER" 53 2>&1 | grep -q "open"; then
    exit 0
fi

# 3. If that also failed, try multiple external domains (upstream DNS working?)
TOP_DOMAINS="google.com reddit.com youtube.com facebook.com amazon.com"
for domain in $TOP_DOMAINS; do
    if timeout $TIMEOUT nslookup "$domain" "$DNS_SERVER" >/dev/null 2>&1; then
        exit 0
    fi
done

# If nothing worked, unhealthy
exit 1

Note that remove the part "Try local domain", if you don't have any local domain.

Then implement this in the docker compose:

    healthcheck:
      test: ["CMD-SHELL", "sh /healthcheck.sh"]
      interval: 30s
      timeout: 10s
      retries: 2
      start_period: 10s

I wrote a full article for that, you can reach it here.

[rvjr]

I can confirm this behaviour as well. I already tried to debug it a little. What I know right now from monitoring everything Uptime Kuma:

• I have approximately 5-10 downtimes a day, at random times, for a duration of around 2 - 20 minutes
• the machine is up and running as it answers ping at all times
• DNS lookups on the upstream servers are running smoothly at all times
• a healthcheck every 15s with dig @127.0.0.1 <test-domain> A +short +retry=1 +timeout=2 (I added dig to the image) shows that the dnsmasq instance in the container is not answering, and they container goes to unhealthy state
• as I have log-queries in the container I can see that no request is logged during the downtimes, not even the healthcheck requests from within the container
• if the container is not being restarted I see logs from healthchecks and clients being answered again after some time
• restarting the container immediately fixes the issue

I am working around this with autoheal now, but there is certainly something wrong, either with dnsmasq itself, or the docker image.

[bokunodev]

i stumbled upon similar issue. in my case dnsmasq always response instantaneously if i send the query through TCP. but it sometimes stuck if it through UDP. i used dog. what strange is that UDP query always success after i send TCP query (possibly fetched the record from cache instead of upstream).
it'd be very helpful if can anyone answer some of my questions.

1. what tool did you use test it? (eg. dig, drill, dog)
2. have you try to send the query through TCP instead of UDP?

thanks

[jganseman]

Same issue. I have, as per previous comments, to the docker-compose.yml added the following lines:

services:
  dnsmasq:
    [...]
    healthcheck:
      test: ["CMD-SHELL", "nslookup google.com >/dev/null 2>&1"]
      interval: 10s
      timeout: 5s
      retries: 3
      start_period: 10s

  autoheal:
    image: willfarrell/autoheal:latest
    container_name: autoheal
    restart: unless-stopped
    environment:
      - AUTOHEAL_CONTAINER_LABEL=autoheal
      - AUTOHEAL_INTERVAL=3
      - AUTOHEAL_START_PERIOD=30
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    logging:
      driver: syslog
      options:
        tag: autoheal

This seems to provide some relief, but when the network is in (heavy) use, autoheal seems to intervene very regularly. On weekend days when the network is barely active, everything goes smooth. On office days, I can be facing a dnsmasq container reboot every 10 minutes or worse.
It seems that either some kind of network traffic, or maybe just a particular volume of traffic at the wrong time, makes the entire thing stall?
docker logs -f autoheal and journalctl -fu docker give me a few high level logging events, but this seems to be a very elusive problem, I haven't been able to debug it further yet...

EDIT 1: I tried to solve the problem by replacing dockurr/dnsmasq with 4km3/docker-dnsmasq . Your favorite AI assistant can help you write a functionally equivalent docker-compose.yml based on your current dockurr/dnsmasq one. After which the following did the trick:

docker compose pull
docker compose down
docker compose up -d
docker compose logs -f dnsmasq

EDIT 2: 4km3/docker-dnsmasq boots much snappier, but has exact the same problem in network_mode:host. The problem must be in dnsmasq itself and/or how docker networking deals with it, not in the containers that run it. Either that, or we all run some common component that trips up dnsmasq in our networks. Anyone else using Wireguard to connect to the network on which dnsmasq runs, or having a Proxmox cluster in their network?