feat: exponential retry decorator

infrastructure/rag/templates/_helpers.tpl

@@ -10,6 +10,10 @@
 {{- printf "%s-usecase-configmap" .Release.Name | trunc 63 | trimSuffix "-" -}}
 {{- end -}}
 
+{{- define "configmap.retryDecoratorName" -}}
+{{- printf "%s-retry-decorator-configmap" .Release.Name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
 {{- define "secret.usecaseName" -}}
 {{- printf "%s-usecase-secret" .Release.Name | trunc 63 | trimSuffix "-" -}}
 {{- end -}}

infrastructure/rag/templates/admin-backend/deployment.yaml

@@ -106,6 +106,8 @@ spec:
               name: {{ template "configmap.keyValueStoreName" . }}
           - configMapRef:
               name: {{ template "configmap.sourceUploaderName" . }}
+          - configMapRef:
+              name: {{ template "configmap.retryDecoratorName" . }}
           - secretRef:
               name: {{ template "secret.langfuseName" . }}
           - secretRef:

infrastructure/rag/templates/backend/deployment.yaml

@@ -131,6 +131,8 @@ spec:
               name: {{ template "configmap.fakeEmbedderName" . }}
           - configMapRef:
               name: {{ template "configmap.chatHistoryName" . }}
+          - configMapRef:
+              name: {{ template "configmap.retryDecoratorName" . }}
           - secretRef:
               name: {{ template "secret.langfuseName" . }}
           - secretRef:

infrastructure/rag/templates/configmap.yaml

@@ -24,3 +24,12 @@ data:
   {{- range $key, $value := .Values.shared.envs.usecase }}
   {{ $key }}: {{ $value | quote }}
   {{- end }}
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: {{ template "configmap.retryDecoratorName" . }}
+data:
+  {{- range $key, $value := .Values.shared.envs.retryDecorator }}
+  {{ $key }}: {{ $value | quote }}
+  {{- end }}

infrastructure/rag/values.yaml

@@ -441,6 +441,14 @@ shared:
     s3:
       S3_ENDPOINT: http://rag-minio:9000
       S3_BUCKET: documents
+    retryDecorator:
+      RETRY_DECORATOR_MAX_RETRIES: "5"
+      RETRY_DECORATOR_RETRY_BASE_DELAY: "0.5"
+      RETRY_DECORATOR_RETRY_MAX_DELAY: "600"
+      RETRY_DECORATOR_BACKOFF_FACTOR: "2"
+      RETRY_DECORATOR_ATTEMPT_CAP: "6"
+      RETRY_DECORATOR_JITTER_MIN: "0.05"
+      RETRY_DECORATOR_JITTER_MAX: "0.25"
     usecase:
 
 

libs/README.md

@@ -18,6 +18,7 @@ It consists of the following python packages:
   - [3.3 Replaceable parts](#33-replaceable-parts)
 - [`4. RAG Core lib`](#4-rag-core-lib)
   - [4.1 Requirements](#41-requirements)
+  - [4.2 Retry decorator (exponential backoff)](#42-retry-decorator-exponential-backoff)
 
 With the exception of the `RAG Core lib` all of these packages contain an API definition and are easy to adjust for your specific use case.
 Each of the packages defines the replaceable parts([1.3 Replaceable Parts](#13-replaceable-parts), [2.3 Replaceable Parts](#23-replaceable-parts), [3.3 Replaceable Parts](#33-replaceable-parts)), expected types and offer a brief description.
@@ -262,3 +263,53 @@ In addition to python libraries the following system packages are required:
 build-essential
 make
 ```
+
+### 4.2 Retry decorator (exponential backoff)
+
+The `rag-core-lib` provides a reusable retry decorator with exponential backoff and rate‑limit awareness for both sync and async functions.
+
+- Module: `rag_core_lib.impl.utils.retry_decorator.retry_with_backoff`
+- Settings: `rag_core_lib.impl.settings.retry_decorator_settings.RetryDecoratorSettings`
+- Works with: synchronous and asynchronous callables
+- Rate-limit aware: optionally inspects HTTP status 429 and headers like `x-ratelimit-reset-requests` / `x-ratelimit-reset-tokens`
+
+Usage example
+
+```python
+from rag_core_lib.impl.utils.retry_decorator import retry_with_backoff
+from rag_core_lib.impl.settings.retry_decorator_settings import RetryDecoratorSettings
+
+# Configure via code (env vars also supported, see below)
+settings = RetryDecoratorSettings(
+    max_retries=3,
+    retry_base_delay=0.2,
+)
+
+@retry_with_backoff(settings=settings)
+def fetch_something():
+    return "ok"
+
+@retry_with_backoff(settings=settings)
+async def fetch_async_something():
+    return "ok"
+```
+
+Configuration
+
+- Environment variables (prefix `RETRY_DECORATOR_`):
+  - `RETRY_DECORATOR_MAX_RETRIES` (default: 5)
+  - `RETRY_DECORATOR_RETRY_BASE_DELAY` (default: 0.5)
+  - `RETRY_DECORATOR_RETRY_MAX_DELAY` (default: 600)
+  - `RETRY_DECORATOR_BACKOFF_FACTOR` (default: 2)
+  - `RETRY_DECORATOR_ATTEMPT_CAP` (default: 6)
+  - `RETRY_DECORATOR_JITTER_MIN` (default: 0.05)
+  - `RETRY_DECORATOR_JITTER_MAX` (default: 0.25)
+
+- Helm chart (shared values): set the same keys under `shared.envs.retryDecorator` in [infrastructure/rag/values.yaml](../infrastructure/rag/values.yaml) to apply cluster‑wide defaults for backend/admin services.
+
+Advanced
+
+- Customize which exceptions trigger retries via `exceptions` and `rate_limit_exceptions` parameters of `retry_with_backoff()`.
+- Header‑based wait: When rate‑limited, the decorator will honor reset headers if present and add jitter.
+
+For more examples, see tests in [./rag-core-lib/tests/retry_decorator_test.py](./rag-core-lib/tests/retry_decorator_test.py).

libs/rag-core-lib/pyproject.toml

@@ -28,6 +28,7 @@ langchain-openai = "^0.3.27"
 [tool.poetry.group.dev.dependencies]
 debugpy = "^1.8.14"
 pytest = "^8.3.5"
+pytest-asyncio = "^0.26.0"
 coverage = "^7.5.4"
 flake8 = "^7.2.0"
 flake8-black = "^0.3.6"

libs/rag-core-lib/src/rag_core_lib/impl/settings/retry_decorator_settings.py

@@ -0,0 +1,83 @@
+"""Module contains settings regarding the STACKIT vLLM."""
+
+from pydantic import Field, PositiveInt, model_validator
+from pydantic_settings import BaseSettings, SettingsConfigDict
+
+
+class RetryDecoratorSettings(BaseSettings):
+    """
+    Contains settings regarding the retry decorator.
+
+    Attributes
+    ----------
+    max_retries : int (> 0)
+        Total retries (not counting the first attempt).
+    retry_base_delay : float (>= 0)
+        Base delay in seconds for the first retry.
+    retry_max_delay : float (> 0)
+        Maximum delay cap for any single wait.
+    backoff_factor : float (>= 1)
+        Exponential backoff factor.
+    attempt_cap : int (>= 0)
+        Cap for exponent growth (backoff_factor ** attempt_cap).
+    jitter_min : float (>= 0)
+        Minimum jitter to add to wait times.
+    jitter_max : float (>= jitter_min)
+        Maximum jitter to add to wait times.
+    """
+
+    model_config = SettingsConfigDict(env_prefix="RETRY_DECORATOR_", case_sensitive=False)
+
+    max_retries: PositiveInt = Field(
+        default=5,
+        title="Max Retries",
+        description="Total retries, not counting the initial attempt.",
+    )
+    retry_base_delay: float = Field(
+        default=0.5,
+        ge=0,
+        title="Retry Base Delay",
+        description="Base delay in seconds for the first retry.",
+    )
+    retry_max_delay: float = Field(
+        default=600.0,
+        gt=0,
+        title="Retry Max Delay",
+        description="Maximum delay cap in seconds for any single wait.",
+    )
+    backoff_factor: float = Field(
+        default=2.0,
+        ge=1.0,
+        title="Backoff Factor",
+        description="Exponential backoff factor (>= 1).",
+    )
+    attempt_cap: int = Field(
+        default=6,
+        ge=0,
+        title="Attempt Cap",
+        description="Cap for exponent growth (backoff_factor ** attempt_cap).",
+    )
+    jitter_min: float = Field(
+        default=0.05,
+        ge=0.0,
+        title="Jitter Min (s)",
+        description="Minimum jitter in seconds.",
+    )
+    jitter_max: float = Field(
+        default=0.25,
+        ge=0.0,
+        title="Jitter Max (s)",
+        description="Maximum jitter in seconds.",
+    )
+
+    @model_validator(mode="after")
+    def _check_relations(self) -> "RetryDecoratorSettings":
+        # Ensure jitter_max >= jitter_min
+        if self.jitter_max < self.jitter_min:
+            raise ValueError("jitter_max must be >= jitter_min")
+        # Ensure retry_max_delay is meaningful vs base
+        if self.retry_max_delay <= 0:
+            raise ValueError("retry_max_delay must be > 0")
+        if self.backoff_factor < 1:
+            raise ValueError("backoff_factor must be >= 1")
+        return self