CLOUDP-314916: OIDC e2e test single cluster (#55)

# Summary

### Test File Summary

- **replica_set_oidc_m2m_user.py**  
- Tests OIDC machine-to-machine (M2M) user authentication for a replica
set.
- Verifies user creation, role updates, access restrictions, and
negative authentication scenarios.

- **replica_set_oidc_m2m_group.py**  
- Tests OIDC group-based (Workload Identity Federation) authentication
in a replica set.
- Covers creation, OIDC provider/role updates, and removal of OIDC
configs and roles.

- **replica_set_oidc_workforce.py**  
- Tests OIDC workforce (human identity) authentication in a replica set.
- Validates user creation and correct automation config for multiple
OIDC providers.

- **sharded_cluster_oidc_m2m_group.py**  
  - Tests OIDC group-based authentication in a sharded cluster.
- Verifies creation, connectivity, provider/role updates, and automation
config state.

- **sharded_cluster_oidc_m2m_user.py**
- Tests OIDC machine-to-machine user authentication in a sharded
cluster, including user creation, role assignment, and
     access restrictions.
- Verifies correct OIDC provider configuration, user propagation, and
negative authentication scenarios.


### Additional PR Changes

- **OIDC Callback Integration:**  
Introduced a custom OIDC callback handler utilizing AWS Cognito for
token acquisition, allowing automated OIDC authentication in E2E tests.

- **Automation Config Tester Improvements:**  
Enhanced assertion helpers to validate OIDC-specific state in the Ops
Manager automation config, including provider counts, configuration
details, and user propagation.

- **New OIDC Fixture Files:**  
Added YAML resource definitions for OIDC-enabled replica sets, sharded
clusters, and MongoDB users, supporting a wide range of authentication
and authorization test cases.

- **Core Controller and Logic Adjustments:**  
Minor changes in Go controller code to ensure robust handling of OIDC
provider configs and roles, and to support expanded test coverage.
  
### AWS Setup:

The project uses AWS Cognito in the mongodb-mms-testing AWS account to
facilitate OIDC authentication testing. This setup includes:


- User Pool: A user pool in Cognito manages the identities.
- Users: We use the user credentials to do authentication.
- App Client: An app client is configured for machine-to-machine (M2M)
authentication.
- Groups: Cognito groups are used to manage users from the user pool for
GroupMembership access.

Environment variables and secrets required for these tests (like client
IDs, URLs, and user IDs, as seen in the Python code) are stored in
Evergreen and fetched from there during test execution.

[Link to the session](http://go/k8s-oidc-session) where I explained the
AWS setup for OIDC
---

## Proof of Work

Added tests are passing.

## Checklist
- [x] Have you linked a jira ticket and/or is the ticket in the title?
- [ ] Have you checked whether your jira ticket required DOCSP changes?
- [ ] Have you checked for release_note changes?

## Reminder (Please remove this when merging)
- Please try to Approve or Reject Changes the PR, keep PRs in review as
short as possible
- Our Short Guide for PRs:
[Link](https://docs.google.com/document/d/1T93KUtdvONq43vfTfUt8l92uo4e4SEEvFbIEKOxGr44/edit?tab=t.0)
- Remember the following Communication Standards - use comment prefixes
for clarity:
  * **blocking**: Must be addressed before approval.
  * **follow-up**: Can be addressed in a later PR or ticket.
  * **q**: Clarifying question.
  * **nit**: Non-blocking suggestions.
  * **note**: Side-note, non-actionable. Example: Praise 
  * --> no prefix is considered a question

---------

Co-authored-by: Maciej Karaś <[email protected]>
Co-authored-by: Lucian Tosa <[email protected]>
Co-authored-by: Lucian Tosa <[email protected]>

Author: 3 people

.evergreen-functions.yml

@@ -1,6 +1,13 @@
 variables:
   - &e2e_include_expansions_in_env
     include_expansions_in_env:
+      - cognito_user_pool_id
+      - cognito_workload_federation_client_id
+      - cognito_user_name
+      - cognito_workload_federation_client_secret
+      - cognito_user_password
+      - cognito_workload_url
+      - cognito_workload_user_id
       - ARTIFACTORY_PASSWORD
       - ARTIFACTORY_USERNAME
       - GRS_PASSWORD

.evergreen-tasks.yml

@@ -1240,6 +1240,32 @@ tasks:
     commands:
       - func: e2e_test
 
+  # OIDC tests
+  - name: e2e_replica_set_oidc_m2m_group
+    tags: [ "patch-run" ]
+    commands:
+      - func: e2e_test
+
+  - name: e2e_replica_set_oidc_m2m_user
+    tags: [ "patch-run" ]
+    commands:
+      - func: e2e_test
+
+  - name: e2e_replica_set_oidc_workforce
+    tags: [ "patch-run" ]
+    commands:
+      - func: e2e_test
+
+  - name: e2e_sharded_cluster_oidc_m2m_group
+    tags: [ "patch-run" ]
+    commands:
+      - func: e2e_test
+
+  - name: e2e_sharded_cluster_oidc_m2m_user
+    tags: [ "patch-run" ]
+    commands:
+      - func: e2e_test
+
   - name: e2e_search_community_basic
     tags: ["patch-run"]
     commands:

.evergreen.yml

@@ -759,6 +759,12 @@ task_groups:
       - e2e_replica_set_pv_resize
       - e2e_sharded_cluster_pv_resize
       - e2e_community_and_meko_replicaset_scale
+      # OIDC test group
+      - e2e_replica_set_oidc_m2m_group
+      - e2e_replica_set_oidc_m2m_user
+      - e2e_replica_set_oidc_workforce
+      - e2e_sharded_cluster_oidc_m2m_group
+      - e2e_sharded_cluster_oidc_m2m_user
     <<: *teardown_group
 
   # this task group contains just a one task, which is smoke testing whether the operator

controllers/om/deployment.go

@@ -639,11 +639,22 @@ func (d Deployment) SetRoles(roles []mdbv1.MongoDbRole) {
 }
 
 func (d Deployment) GetRoles() []mdbv1.MongoDbRole {
-	val, ok := d["roles"].([]mdbv1.MongoDbRole)
-	if !ok {
+	roles, ok := d["roles"]
+	if !ok || roles == nil {
 		return []mdbv1.MongoDbRole{}
 	}
-	return val
+
+	rolesBytes, err := json.Marshal(roles)
+	if err != nil {
+		return []mdbv1.MongoDbRole{}
+	}
+
+	var result []mdbv1.MongoDbRole
+	if err := json.Unmarshal(rolesBytes, &result); err != nil {
+		return []mdbv1.MongoDbRole{}
+	}
+
+	return result
 }
 
 // GetAgentVersion returns the current version of all Agents in the deployment. It's empty until the

controllers/operator/authentication/authentication.go

@@ -264,6 +264,7 @@ func enableAgentAuthentication(conn om.Connection, opts Options, log *zap.Sugare
 
 	// we then configure the agent authentication for that type
 	mechanism := convertToMechanismOrPanic(opts.AgentMechanism, ac)
+
 	if err := ensureAgentAuthenticationIsConfigured(conn, opts, ac, mechanism, log); err != nil {
 		return xerrors.Errorf("error ensuring agent authentication is configured: %w", err)
 	}

docker/mongodb-kubernetes-tests/kubetester/automation_config_tester.py

@@ -97,6 +97,18 @@ def assert_processes_size(self, expected_size: int):
     def assert_sharding_size(self, expected_size: int):
         assert len(self.automation_config["sharding"]) == expected_size
 
+    def assert_oidc_providers_size(self, expected_size: int):
+        assert len(self.automation_config["oidcProviderConfigs"]) == expected_size
+
+    def assert_oidc_configuration(self, oidc_config: Optional[Dict] = None):
+        actual_configs = self.automation_config["oidcProviderConfigs"]
+        assert len(actual_configs) == len(
+            oidc_config
+        ), f"Expected {len(oidc_config)} OIDC configs, but got {len(actual_configs)}"
+
+        for expected, actual in zip(oidc_config, actual_configs):
+            assert expected == actual, f"Expected OIDC config: {expected}, but got: {actual}"
+
     def assert_empty(self):
         self.assert_processes_size(0)
         self.assert_replica_sets_size(0)

docker/mongodb-kubernetes-tests/kubetester/mongodb.py

@@ -404,6 +404,41 @@ def get_authentication(self) -> Optional[Dict]:
         except KeyError:
             return {}
 
+    def get_oidc_provider_configs(self) -> Optional[Dict]:
+        try:
+            return self["spec"]["security"]["authentication"]["oidcProviderConfigs"]
+        except KeyError:
+            return {}
+
+    def set_oidc_provider_configs(self, oidc_provider_configs: Dict):
+        self["spec"]["security"]["authentication"]["oidcProviderConfigs"] = oidc_provider_configs
+        return self
+
+    def append_oidc_provider_config(self, new_config: Dict):
+        if "oidcProviderConfigs" not in self["spec"]["security"]["authentication"]:
+            self["spec"]["security"]["authentication"]["oidcProviderConfigs"] = []
+
+        oidc_configs = self["spec"]["security"]["authentication"]["oidcProviderConfigs"]
+
+        oidc_configs.append(new_config)
+
+        self["spec"]["security"]["authentication"]["oidcProviderConfigs"] = oidc_configs
+
+        return self
+
+    def get_roles(self) -> Optional[Dict]:
+        try:
+            return self["spec"]["security"]["roles"]
+        except KeyError:
+            return {}
+
+    def append_role(self, new_role: Dict):
+        if "roles" not in self["spec"]["security"]:
+            self["spec"]["security"]["roles"] = []
+        self["spec"]["security"]["roles"].append(new_role)
+
+        return self
+
     def get_authentication_modes(self) -> Optional[Dict]:
         try:
             return self.get_authentication()["modes"]

docker/mongodb-kubernetes-tests/kubetester/mongotester.py

@@ -1,6 +1,7 @@
 import copy
 import inspect
 import logging
+import os
 import random
 import string
 import threading
@@ -11,6 +12,8 @@
 from kubetester import kubetester
 from kubetester.kubetester import KubernetesTester
 from opentelemetry import trace
+from pycognito import Cognito
+from pymongo.auth_oidc import OIDCCallback, OIDCCallbackContext, OIDCCallbackResult
 from pymongo.errors import OperationFailure, PyMongoError, ServerSelectionTimeoutError
 from pytest import fail
 
@@ -61,6 +64,18 @@ def with_ldap(ssl_certfile: Optional[str] = None, tls_ca_file: Optional[str] = N
     return options
 
 
+class MyOIDCCallback(OIDCCallback):
+    def fetch(self, context: OIDCCallbackContext) -> OIDCCallbackResult:
+        u = Cognito(
+            user_pool_id=os.getenv("cognito_user_pool_id"),
+            client_id=os.getenv("cognito_workload_federation_client_id"),
+            username=os.getenv("cognito_user_name"),
+            client_secret=os.getenv("cognito_workload_federation_client_secret"),
+        )
+        u.authenticate(password=os.getenv("cognito_user_password"))
+        return OIDCCallbackResult(access_token=u.id_token)
+
+
 class MongoTester:
     """MongoTester is a general abstraction to work with mongo database. It encapsulates the client created in
     the constructor. All general methods non-specific to types of mongodb topologies should reside here."""
@@ -277,6 +292,47 @@ def assert_ldap_authentication(
                     fail(msg=f"unable to authenticate after {total_attempts} attempts")
                 time.sleep(5)
 
+    def assert_oidc_authentication(
+        self,
+        db: str = "admin",
+        collection: str = "myCol",
+        attempts: int = 10,
+    ):
+        assert attempts > 0
+
+        props = {"OIDC_CALLBACK": MyOIDCCallback()}
+
+        total_attempts = attempts
+        while True:
+            attempts -= 1
+            try:
+                # Initialize the MongoDB client with OIDC authentication
+                self.client = self._init_client(
+                    authMechanism="MONGODB-OIDC",
+                    authMechanismProperties=props,
+                )
+                # Perform a write operation to test authentication
+                self.client[db][collection].insert_one({"test": "oidc_auth_test"})
+                return
+            except OperationFailure as e:
+                if attempts == 0:
+                    raise RuntimeError(f"Unable to authenticate after {total_attempts} attempts: {e}")
+                time.sleep(5)
+
+    def assert_oidc_authentication_fails(self, db: str = "admin", collection: str = "myCol", attempts: int = 10):
+        assert attempts > 0
+        total_attempts = attempts
+        while True:
+            attempts -= 1
+            try:
+                if attempts <= 0:
+                    fail(msg=f"was able to authenticate with OIDC after {total_attempts} attempts")
+
+                self.assert_oidc_authentication(db, collection, 1)
+                time.sleep(5)
+            except RuntimeError:
+                return
+
     def upload_random_data(
         self,
         count: int,

docker/mongodb-kubernetes-tests/kubetester/oidc.py

@@ -0,0 +1,26 @@
+import os
+
+# Note: The project uses AWS Cognito in the mongodb-mms-testing AWS account to facilitate OIDC authentication testing.
+# This setup includes:
+
+# User Pool: A user pool in Cognito manages the identities.
+# Users: We use the user credentials to do authentication.
+# App Client: An app client is configured for machine-to-machine (M2M) authentication.
+# Groups: Cognito groups are used to manage users from the user pool for GroupMembership access.
+
+# Environment variables and secrets required for these tests (like client IDs, URLs, and user IDs, as seen in the Python code)
+# are stored in Evergreen and fetched from there during test execution.
+
+# A session explaining the setup can be found here: http://go/k8s-oidc-session
+
+
+def get_cognito_workload_client_id() -> str:
+    return os.getenv("cognito_workload_federation_client_id", "")
+
+
+def get_cognito_workload_url() -> str:
+    return os.getenv("cognito_workload_url", "")
+
+
+def get_cognito_workload_user_id() -> str:
+    return os.getenv("cognito_workload_user_id", "")

docker/mongodb-kubernetes-tests/tests/authentication/fixtures/oidc/oidc-user.yaml

@@ -0,0 +1,13 @@
+---
+apiVersion: mongodb.com/v1
+kind: MongoDBUser
+metadata:
+  name: oidc-user-0
+spec:
+  username: "<filled-in-test>"
+  db: "$external"
+  mongodbResourceRef:
+    name: oidc-replica-set
+  roles:
+    - db: "admin"
+      name: "readWriteAnyDatabase"