airflow.exceptions.AirflowException: No module named 'airflow.api.auth.backend.ldap_auth'

[manel00]

Apache Airflow version

2.10.0

If "Other Airflow 2 version" selected, which one?

No response

What happened?

LDAP does not work

What you think should happen instead?

No response

How to reproduce

Deploy airflow on Docker

Operating System

Ubuntu 22

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[kunaljubce]

@manel00 Can you please provide some details on this?

• What is the exact error message? Simply mentioning "LDAP does not work" is not sufficient.
• Can you provide the error trace or logs?
• How can anyone reproduce it? You have mentioned "Deploy Airflow on Docker", which isn't sufficient. Any specific configs you are passing to setup LDAP?
• What do you think should happen instead of the failure you're facing?

[potiuk]

Apparently (other discussion) it was a bug in script installing airflow where ldap extra was not specified after uninstalling all packages.

[ipesanz]

Hello,

I faced the same error. Providing my detailed installation of Airflow and dependencies:

Installation of airflow via pip

# As per: 17-Sept-24
AIRFLOW_VERSION=2.10.1
PYTHON_VERSION=3.10  #hardcoded, as per Ubuntu 22 installation

pip install "apache-airflow[celery]==2.10.1" --constraint "https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
pip install psycopg2-binary apache-airflow[ldap]

I just proceeded to install my LDAP configuration for this instance of airflow (first timer), and when running the webserver it´s not starting due to the lack of dependencies.

Links used for my configuration:

# REF: https://www.restack.io/docs/airflow-knowledge-airflow-ldap-integration
# REF: https://github.com/astronomer/airflow-ldap-example/blob/main/webserver_config.py
# REF: https://artifacthub.io/packages/helm/airflow-helm/airflow/8.0.2

Modifications done in airflow.cfg

sudo sed -i "s|^auth_backends = .*|auth_backends = airflow.api.auth.backend.ldap_auth|" $AIRFLOW_HOME/airflow.cfg
sudo sed -i "s|AUTH_TYPE = AUTH_DB|AUTH_TYPE = AUTH_LDAP|" $AIRFLOW_HOME/webserver_config.py
cat <<EOF >> $AIRFLOW_HOME/webserver_config.py
from airflow import configuration as conf

SQLALCHEMY_DATABASE_URI = conf.get("core", "SQL_ALCHEMY_CONN")
CSRF_ENABLED = True

AUTH_ROLE_ADMIN = "Admin"
[ . . . ] More configs of LDAP in the webserver_config

[ipesanz]

Error trace

airflow@aHiddenServer:~/airflow$ airflow webserver
/usr/local/lib/python3.10/dist-packages/airflow/configuration.py:765 FutureWarning: The auth_backends setting in [api] has had airflow.api.auth.backend.session added in the running config, which is needed by the UI. Please update your config before Apache Airflow 3.0.
  ____________       _____________
 ____    |__( )_________  __/__  /________      __
____  /| |_  /__  ___/_  /_ __  /_  __ \_ | /| / /
___  ___ |  / _  /   _  __/ _  / / /_/ /_ |/ |/ /
 _/_/  |_/_/  /_/    /_/    /_/  \____/____/|__/
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
Access Logformat:
=================================================================
/usr/local/lib/python3.10/dist-packages/airflow/configuration.py:2121 FutureWarning: section/key [core/sql_alchemy_conn] has been deprecated, you should use[database/sql_alchemy_conn] instead. Please update your `conf.get*` call to use the new name
[2024-10-08T11:43:54.682+0200] {init_security.py:65} CRITICAL - Cannot import airflow.api.auth.backend.ldap_auth for API authentication due to: No module named 'airflow.api.auth.backend.ldap_auth'
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/airflow/www/extensions/init_security.py", line 61, in init_api_experimental_auth
    auth = import_module(backend.strip())
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'airflow.api.auth.backend.ldap_auth'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/__main__.py", line 10, in <module>
    run(prog="gunicorn")
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/app/wsgiapp.py", line 66, in run
    WSGIApplication("%(prog)s [OPTIONS] [APP_MODULE]", prog=prog).run()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/app/base.py", line 235, in run
    super().run()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/app/base.py", line 71, in run
    Arbiter(self).run()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/arbiter.py", line 57, in __init__
    self.setup(app)
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/arbiter.py", line 117, in setup
    self.app.wsgi()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/app/base.py", line 66, in wsgi
    self.callable = self.load()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/app/wsgiapp.py", line 57, in load
    return self.load_wsgiapp()
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/app/wsgiapp.py", line 47, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/usr/local/lib/python3.10/dist-packages/gunicorn/util.py", line 423, in import_app
    app = app(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/airflow/www/app.py", line 196, in cached_app
    app = create_app(config=config, testing=testing)
  File "/usr/local/lib/python3.10/dist-packages/airflow/www/app.py", line 153, in create_app
    init_api_experimental_auth(flask_app)
  File "/usr/local/lib/python3.10/dist-packages/airflow/www/extensions/init_security.py", line 66, in init_api_experimental_auth
    raise AirflowException(err)
airflow.exceptions.AirflowException: No module named 'airflow.api.auth.backend.ldap_auth'
[2024-10-08T11:45:53.926+0200] {webserver_command.py:223} ERROR - No response from gunicorn master within 120 seconds
[2024-10-08T11:45:53.927+0200] {webserver_command.py:224} ERROR - Shutting down webserver

[potiuk]

well. If you do not install ldap "extra" - expecting that it will be installed magically is kind of strange. Why do you think ldap should be there if you have not used ldap extra to install airflow @ipesanz ? Or maybe you have not read and understood https://airflow.apache.org/docs/apache-airflow/stable/extra-packages-ref.html ?