feat: use DAB to load dynaconf settings (#1223)

# [AAP-37673] EDA: Use DAB:Dynaconf to load Django static settings

## Description
<!-- Mandatory: Provide a clear, concise description of the changes and
their purpose -->
It is the organization's effect to load settings using the same method. 

EDA used to define all settings and expose to Django settings in
`aap_eda/settings/default.py`. We continue to use this file to load and
expose settings through DAB:Dynaconf. We move individual settings from
this file to other files:
* all settings that can be overwritten are moved to
`aap_eda/settings/defaults.py`
* all internal used settings that are not exposed for overwriting are
moved to `aap_eda/settings/core.py`
* all computed or adjusted settings and logic are moved to
`aap_eda/settings/post_load.py`
* redis specific settings preparations are moved to
`aap_eda/settings/redis.py`

Closes AAP-37673
Jira: [AAP-37673](https://jira.aap.com/browse/AAP-37673) 

## Type of Change
<!-- Mandatory: Check one or more boxes that apply -->
- [ ] Bug fix (non-breaking change which fixes an issue)
- [x] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to not work as expected)
- [ ] Documentation update
- [ ] Test update
- [ ] Refactoring (no functional changes)
- [ ] Development environment change
- [ ] Configuration change
- [ ] CI change

## Self-Review Checklist
<!-- These items help ensure quality - they complement our automated CI
checks -->
- [ ] I have performed a self-review of my code
- [ ] I have added relevant comments to complex code sections
- [ ] I have updated documentation where needed
- [ ] I have considered the security impact of these changes
- [ ] I have considered performance implications
- [ ] I have thought about error handling and edge cases
- [ ] I have tested the changes in my local environment
- [ ] I have run the linters and test suite locally
- [ ] I have tested the changes on integration environment

## Testing Instructions
<!-- Optional for test-only changes. Mandatory for all other changes -->
<!-- Must be detailed enough for reviewers to reproduce -->
### Prerequisites
<!-- List any specific setup required -->

### Steps to Test

1.
2.
3.

### Expected Results
<!-- Describe what should happen after following the steps -->

## Additional Context
<!-- Optional but helpful information -->

### Required Actions
<!-- Check if changes require work in other areas -->
<!-- Remove section if no external actions needed -->
- [ ] Requires documentation updates
  <!-- API docs, feature docs, deployment guides -->
- [ ] Requires downstream repository changes
  <!-- Specify repos: django-ansible-base, eda-server, etc. -->
- [ ] Requires infrastructure/deployment changes
  <!-- CI/CD, installer updates, new services -->
- [ ] Requires coordination with other teams
  <!-- UI team, platform services, infrastructure -->
- [ ] Blocked by PR/MR: #XXX |
<https://github.com/example/repo/pull/XXX>
  <!-- Reference blocking PRs/MRs with brief context -->

### Screenshots/Logs
<!-- Add if relevant to demonstrate the changes -->

Author: bzwei

Taskfile.dist.yaml

@@ -1,7 +1,8 @@
 version: "3"
 
 env:
-  DJANGO_SETTINGS_MODULE: "aap_eda.settings.development"
+  EDA_MODE: development
+  DJANGO_SETTINGS_MODULE: "aap_eda.settings.default"
 
 vars:
   DOCKER_COMPOSE: '{{ default "docker-compose" .DOCKER_COMPOSE }}'

poetry.lock

@@ -36,7 +36,6 @@ dev = ["psycopg-binary"]
 python = ">=3.11,<3.13"
 django = ">=4.2,<4.3"
 djangorestframework = "3.15.*"
-dynaconf = ">=3.1.12,<3.3"
 drf-spectacular = ">=0.26.5,<0.27"
 channels = { version = "4.0.*", extras = ["daphne"] }
 psycopg-binary = { version = "*", optional = true }
@@ -48,7 +47,7 @@ cryptography = ">=42,<43"
 kubernetes = "26.1.*"
 podman = "5.4.*"
 rq-scheduler = "^0.10"
-django-ansible-base = { git = "https://github.com/ansible/django-ansible-base.git", tag = "2025.1.31", extras = [
+django-ansible-base = { git = "https://github.com/ansible/django-ansible-base.git", tag = "2025.3.7", extras = [
     "channel-auth",
     "rbac",
     "redis-client",
@@ -72,6 +71,7 @@ django-flags = "^5.0.13"
 
 [tool.poetry.group.test.dependencies]
 pytest = "*"
+pytest-env = "*"
 pytest-django = "*"
 pytest-asyncio = "*"
 requests = { version = "*", python = "<4.0" }

pyproject.toml

@@ -1,4 +1,6 @@
 [pytest]
 asyncio_mode = auto
-DJANGO_SETTINGS_MODULE = aap_eda.settings.development
+env =
+    EDA_MODE=development
+DJANGO_SETTINGS_MODULE = aap_eda.settings.default
 log_file_level = INFO

pytest.ini

@@ -22,7 +22,7 @@
 from django.conf import settings
 from rq import results as rq_results
 
-from aap_eda.settings import default
+from aap_eda.settings import core as core_settings, redis as redis_settings
 
 __all__ = [
     "Job",
@@ -131,7 +131,7 @@ def _prune_redis_kwargs(**kwargs) -> dict[str, typing.Any]:
     db = kwargs.get("db", None)
     if (db is not None) and (kwargs.get("mode", "") == "cluster"):
         del kwargs["db"]
-        if db != default.DEFAULT_REDIS_DB:
+        if db != core_settings.DEFAULT_REDIS_DB:
             logger.info(
                 f"clustered redis supports only the default db"
                 f"; db specified: {db}"
@@ -151,7 +151,7 @@ def get_redis_client(**kwargs) -> typing.Union[DABRedis, DABRedisCluster]:
 
 def get_redis_status() -> dict:
     """Query DAB for the status of Redis."""
-    kwargs = default.rq_redis_client_instantiation_parameters()
+    kwargs = redis_settings.rq_redis_client_instantiation_parameters()
     kwargs = _prune_redis_kwargs(**kwargs)
     return _get_redis_status(_create_url_from_parameters(**kwargs), **kwargs)
 
@@ -329,7 +329,7 @@ def _get_necessary_client_connection(
 ) -> rq.Connection:
     if not isinstance(connection, (DABRedis, DABRedisCluster)):
         connection = get_redis_client(
-            **default.rq_redis_client_instantiation_parameters()
+            **redis_settings.rq_redis_client_instantiation_parameters()
         )
     return connection
 

src/aap_eda/core/tasking/__init__.py

@@ -0,0 +1,15 @@
+#  Copyright 2025 Red Hat, Inc.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+from .default import *  # noqa: F403

src/aap_eda/settings/__init__.py

@@ -0,0 +1,184 @@
+#  Copyright 2025 Red Hat, Inc.
+#
+#  Licensed under the Apache License, Version 2.0 (the "License");
+#  you may not use this file except in compliance with the License.
+#  You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+#  Unless required by applicable law or agreed to in writing, software
+#  distributed under the License is distributed on an "AS IS" BASIS,
+#  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#  See the License for the specific language governing permissions and
+#  limitations under the License.
+
+# Define all settings use internally, not exposed to users for overwriting.
+
+INSTALLED_APPS = [
+    "daphne",
+    "flags",
+    # Django apps
+    "django.contrib.auth",
+    "django.contrib.contenttypes",
+    "django.contrib.sessions",
+    "django.contrib.staticfiles",
+    # Third party apps
+    "rest_framework",
+    "drf_spectacular",
+    "django_rq",
+    "django_filters",
+    "ansible_base.rbac",
+    "ansible_base.resource_registry",
+    "ansible_base.jwt_consumer",
+    "ansible_base.rest_filters",
+    "ansible_base.feature_flags",
+    # Local apps
+    "aap_eda.api",
+    "aap_eda.core",
+]
+
+
+MIDDLEWARE = [
+    "django.middleware.security.SecurityMiddleware",
+    "django.contrib.sessions.middleware.SessionMiddleware",
+    "django.middleware.locale.LocaleMiddleware",
+    "django.middleware.common.CommonMiddleware",
+    "django.middleware.csrf.CsrfViewMiddleware",
+    "django.contrib.auth.middleware.AuthenticationMiddleware",
+    "django.middleware.clickjacking.XFrameOptionsMiddleware",
+    "crum.CurrentRequestUserMiddleware",
+]
+
+ROOT_URLCONF = "aap_eda.urls"
+
+TEMPLATES = [
+    {
+        "BACKEND": "django.template.backends.django.DjangoTemplates",
+        "DIRS": [],
+        "APP_DIRS": True,
+        "OPTIONS": {
+            "context_processors": [
+                "django.template.context_processors.debug",
+                "django.template.context_processors.request",
+                "django.contrib.auth.context_processors.auth",
+            ],
+        },
+    },
+]
+
+WSGI_APPLICATION = "aap_eda.wsgi.application"
+
+ASGI_APPLICATION = "aap_eda.asgi.application"
+
+# Password validation
+# https://docs.djangoproject.com/en/4.1/ref/settings/#auth-password-validators
+
+AUTH_PASSWORD_VALIDATORS = [
+    {
+        "NAME": "django.contrib.auth.password_validation.UserAttributeSimilarityValidator",  # noqa: E501
+    },
+    {
+        "NAME": "django.contrib.auth.password_validation.MinimumLengthValidator",  # noqa: E501
+    },
+    {
+        "NAME": "django.contrib.auth.password_validation.CommonPasswordValidator",  # noqa: E501
+    },
+    {
+        "NAME": "django.contrib.auth.password_validation.NumericPasswordValidator",  # noqa: E501
+    },
+]
+
+# Internationalization
+# https://docs.djangoproject.com/en/4.1/topics/i18n/
+
+LANGUAGE_CODE = "en-us"
+
+USE_I18N = True
+
+TIME_ZONE = "UTC"
+
+USE_TZ = True
+
+# Default primary key field type
+# https://docs.djangoproject.com/en/4.1/ref/settings/#default-auto-field
+DEFAULT_AUTO_FIELD = "django.db.models.BigAutoField"
+
+AUTH_USER_MODEL = "core.User"
+
+REST_FRAMEWORK = {
+    "DEFAULT_SCHEMA_CLASS": "drf_spectacular.openapi.AutoSchema",
+    "DEFAULT_PAGINATION_CLASS": "aap_eda.api.pagination.DefaultPagination",
+    "PAGE_SIZE": 20,
+    "DEFAULT_AUTHENTICATION_CLASSES": [
+        "aap_eda.api.authentication.SessionAuthentication",
+        "rest_framework.authentication.BasicAuthentication",
+        "aap_eda.api.authentication.WebsocketJWTAuthentication",
+        "ansible_base.jwt_consumer.eda.auth.EDAJWTAuthentication",
+    ],
+    "DEFAULT_PERMISSION_CLASSES": [
+        "rest_framework.permissions.IsAuthenticated",
+        "ansible_base.rbac.api.permissions.AnsibleBaseObjectPermissions",
+    ],
+    "TEST_REQUEST_DEFAULT_FORMAT": "json",
+    "DEFAULT_METADATA_CLASS": "aap_eda.api.metadata.EDAMetadata",
+    "EXCEPTION_HANDLER": "aap_eda.api.exceptions.api_fallback_handler",
+}
+
+DEFAULT_REDIS_DB = 0
+
+# ---------------------------------------------------------
+# TASKING SETTINGS
+# ---------------------------------------------------------
+RQ = {
+    "JOB_CLASS": "aap_eda.core.tasking.Job",
+    "QUEUE_CLASS": "aap_eda.core.tasking.Queue",
+    "SCHEDULER_CLASS": "aap_eda.core.tasking.Scheduler",
+    "WORKER_CLASS": "aap_eda.core.tasking.Worker",
+}
+
+# Time window in seconds to consider a worker as dead
+DEFAULT_WORKER_HEARTBEAT_TIMEOUT = 60
+DEFAULT_WORKER_TTL = 5
+
+RQ_STARTUP_JOBS = []
+
+# Id of the scheduler job it's required when we have multiple instances of
+# the scheduler running to avoid duplicate jobs
+RQ_PERIODIC_JOBS = [
+    {
+        "func": (
+            "aap_eda.tasks.orchestrator.enqueue_monitor_rulebook_processes"
+        ),
+        "interval": 5,
+        "id": "enqueue_monitor_rulebook_processes",
+    },
+    {
+        "func": "aap_eda.tasks.project.monitor_project_tasks",
+        "interval": 30,
+        "id": "monitor_project_tasks",
+    },
+]
+RQ_CRON_JOBS = []
+
+ANSIBLE_BASE_CUSTOM_VIEW_PARENT = "aap_eda.api.views.dab_base.BaseAPIView"
+
+# ---------------------------------------------------------
+# DJANGO ANSIBLE BASE RESOURCES REGISTRY SETTINGS
+# ---------------------------------------------------------
+ANSIBLE_BASE_RESOURCE_CONFIG_MODULE = "aap_eda.api.resource_api"
+
+# ---------------------------------------------------------
+# DJANGO ANSIBLE BASE RBAC SETTINGS
+# ---------------------------------------------------------
+DEFAULT_ORGANIZATION_NAME = "Default"
+ANSIBLE_BASE_SERVICE_PREFIX = "eda"
+ANSIBLE_BASE_TEAM_MODEL = "core.Team"
+ANSIBLE_BASE_ORGANIZATION_MODEL = "core.Organization"
+
+# Organization and object roles will come from create_initial_data
+ANSIBLE_BASE_ROLE_PRECREATE = {}
+
+ANSIBLE_BASE_ALLOW_SINGLETON_USER_ROLES = True
+ANSIBLE_BASE_CHECK_RELATED_PERMISSIONS = ["view"]
+
+DEFAULT_SYSTEM_PG_NOTIFY_CREDENTIAL_NAME = "_DEFAULT_EDA_PG_NOTIFY_CREDS"