celery worker stops working

[mertcangokgoz]

Self-Hosted Version

25.8.0

CPU Architecture

x86_64

Docker Version

28.3.3

Docker Compose Version

v2.39.1

Machine Specification

• My system meets the minimum system requirements of Sentry

We are running Sentry on a dedicated server.

Specs

CPU: Intel Core i9
RAM: 64 GB

Steps to Reproduce

• N/A

Expected Result

We have a self-hosted Sentry server that we use for 40 projects. Due to the high volume of events coming in, our Redis memory usage is very high, which causes the machine running Sentry to become unreachable due to RAM usage. To prevent this, we added a memory limit to Redis.

However, Celery occasionally stops working and gives the following error. Even when the workers are restarted, this error continues to occur.

responseError("OOM command not allowed when used memory > 'maxmemory'.")

As a temporary solution, it starts working when I take the following action.

docker compose exec redis redis-cli flushall && docker compose restart worker

my redis.conf

# redis.conf

# The 'maxmemory' directive controls the maximum amount of memory Redis is allowed to use.
# Setting 'maxmemory 0' means there is no limit on memory usage, allowing Redis to use as much
# memory as the operating system allows. This is suitable for environments where memory
# constraints are not a concern.
#
# Alternatively, you can specify a limit, such as 'maxmemory 15gb', to restrict Redis to
# using a maximum of 15 gigabytes of memory.
#
# Example:
# maxmemory 0         # Unlimited memory usage
maxmemory 40gb


# This setting determines how Redis evicts keys when it reaches the memory limit.
# `allkeys-lru` evicts the least recently used keys from all keys stored in Redis,
# allowing frequently accessed data to remain in memory while older data is removed.
# That said we use `volatile-lru` as Redis is used both as a cache and processing
# queue in self-hosted Sentry.
# > The volatile-lru and volatile-random policies are mainly useful when you want to
# > use a single Redis instance for both caching and for a set of persistent keys.
# > However, you should consider running two separate Redis instances in a case like
# > this, if possible.

maxmemory-policy volatile-random

Actual Result

• N/A

Event ID

No response

[aldy505]

We're aware of the Redis issue causing OOMs here and there and everywhere. My recommendation would be removing the maxmemory, and executing docker compose exec redis redis-cli flushall every night (or a few hours after OOM) using cron.

I know this is not a good solution. But I want to keep your Sentry instance running normally.

You may also replace it with Valkey, it's 100% compatible. #3868

[mertcangokgoz]

We're aware of the Redis issue causing OOMs here and there and everywhere. My recommendation would be removing the maxmemory, and executing docker compose exec redis redis-cli flushall every night (or a few hours after OOM) using cron.

I know this is not a good solution. But I want to keep your Sentry instance running normally.

You may also replace it with Valkey, it's 100% compatible. #3868

Thank you for your response. I am aware that we are experiencing issues with Redis due to too many projects and excessive data flowing from them. I hope Valkey will solve our problem. I will do my tests and report back from here.