Airflow webUI "This application is down for maintenance" . for dags with huge number of tasks

[kaveti]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.5.3

What happened?

I have dag which dynamically generates close to 800 tasks. when i hit /tries?day=30 and /duration?days=30 webserver API.

it just simply fails with error.

I have tried the following things.

• Gunicorn Workers: Increasing the number of workers has not resolved the page loading issues, as the problem isn’t related to incoming request traffic.
• Timeout Settings: Adjusted both WORKER_TIMEOUT (3000) and master WORKER_TIMEOUT(3000) to higher values, but the page still fails to load.
• Worker Class: Switching to the gevent worker class did not improve performance, despite its ability to handle parallelism.
• CPU Limits: Increased the webserver’s CPU limits to 24 cores, yet one of the Gunicorn worker consistently maxes out CPU usage.

note: I have checked the webserver_error.log as well. no worker is getting killed abruptly.

What you think should happen instead?

No response

How to reproduce

• Create a dag which generation dynamic tasks more than 800. and then hit /dag_id/tries?day=30 or /dag_id/duration?day=30 API.

Operating System

Red Hat Enterprise Linux (9.4) - rhel

Versions of Apache Airflow Providers

No response

Deployment

Other 3rd-party Helm chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

There were lots of similar issues and optimisations implemented since 2.5.3. I recommend you to upgrade it to later version (2.5.3 is released 1.5 year ago and 2.10.2 has proably more than 1000 fixes since).

Please upgrade and try again.

Also if after upgrade you see the same issue - in order to see what happens, some anaysis of what happens in the proces would be useful. Running pyspy on the process of gunicorn could bring more issues. Until we get some of those details - converting it into a discussion, maybe someone with similar behaviour could chime in.