Use the Kubernetes Ansible collection (operator-framework#2646)

* Use the Kubernetes Ansible collection

Author: fabianvf

CHANGELOG.md

@@ -19,6 +19,7 @@
 
 ### Changed
 - Ansible scaffolding has been rewritten to be simpler and make use of newer features of Ansible and Molecule.
+    - The Kubernetes modules have migrated to the [Kubernetes Ansible collection](https://github.com/ansible-collections/kubernetes). All scaffolded code now references modules from this collection instead of Ansible Core. No immediate action is required for existing users of the modules from core, though it is recommended they switch to using the collection to continue to get non-critical bugfixes and features. The collection is now installed by default in the base image. ([#2646](https://github.com/operator-framework/operator-sdk/pull/2646))
     - No longer generates the build/test-framework directory or molecule/test-cluster scenario
     - Adds new `cluster` scenario that can be used to test against an existing cluster
     - There is no longer any Ansible templating done in the `deploy/` directory, any templates used for testing will be located in `molecule/templates/` instead.

doc/ansible/dev/apb_migration_guide.md

@@ -155,7 +155,7 @@ would become:
 
 ```yaml
 - name: Create bind credential secret
-  k8s:
+  community.kubernetes.k8s:
     definition:
       apiVersion: v1
       kind: Secret
@@ -171,7 +171,7 @@ would become:
         DB_NAME: "{{ postgresql_database | b64encode }}"
 
 - name: Attach secret to CR status
-  k8s_status:
+  operator_sdk.util.k8s_status:
     api_version: apps.example.com/v1alpha1
     kind: PostgreSQL
     name: '{{ meta.name }}'
@@ -182,8 +182,8 @@ would become:
 
 ### ansible_kubernetes_modules
 The `ansible_kubernetes_modules` role and the generated modules are now deprecated.
-The `k8s` module was added in Ansible 2.6 and is the supported way to interact with Kubernetes from Ansible.
-The `k8s` module takes normal kubernetes manifests, so if you currently rely on the old generated modules some refactoring will be required.
+The  `community.kubernetes` Ansible collection supports Ansible 2.9+ and is the supported way to interact with Kubernetes from Ansible.
+The `community.kubernetes.k8s` module takes normal kubernetes manifests, so if you currently rely on the old generated modules some refactoring will be required.
 
 
 # convert.py

doc/ansible/dev/developer_guide.md

@@ -19,7 +19,7 @@ and test them using a playbook.
 
 ### Installing the k8s Ansible modules
 
-To install the k8s Ansible modules, one must first install Ansible 2.6+. On
+To install the k8s Ansible modules, one must first install Ansible 2.9+. On
 Fedora/Centos:
 ```bash
 $ sudo dnf install ansible
@@ -31,6 +31,11 @@ Python][openshift_restclient_python] package. This can be installed from pip:
 $ pip3 install openshift
 ```
 
+Finally, a user must install the Ansible Kubernetes collection from ansible-galaxy:
+```bash
+$ ansible-galaxy collection install community.kubernetes
+```
+
 ### Testing the k8s Ansible modules locally
 
 Sometimes it is beneficial for a developer to run the Ansible code from their
@@ -61,7 +66,7 @@ we will create and delete a namespace with the switch of a variable:
 ```yaml
 ---
 - name: set example-memcached namespace to {{ state }}
-  k8s:
+  community.kubernetes.k8s:
     api_version: v1
     kind: Namespace
     name: example-memcached

doc/ansible/dev/retroactively-owned-resources.md

@@ -133,7 +133,7 @@ This file can be used as-is without user adjustments.
     - name: Import user variables
       include_vars: vars.yml
     - name: Retrieve owning resource
-      k8s_info:
+      community.kubernetes.k8s_info:
         api_version: "{{ owning_resource.apiVersion }}"
         kind: "{{ owning_resource.kind }}"
         name: "{{ owning_resource.name }}"
@@ -144,7 +144,7 @@ This file can be used as-is without user adjustments.
       include_tasks: each_resource.yml
       loop: "{{ resources_to_own }}"
       vars:
-        to_be_owned: '{{ q("k8s",
+        to_be_owned: '{{ q("community.kubernetes.k8s",
           api_version=item.apiVersion,
           kind=item.kind,
           resource_name=item.name,
@@ -168,7 +168,7 @@ This file can be used as-is without user adjustments.
     - to_be_owned.metadata.namespace == owning_resource.namespace
     - (to_be_owned.metadata.ownerReferences is not defined) or
       (owner_reference not in to_be_owned.metadata.ownerReferences)
-  k8s:
+  community.kubernetes.k8s:
     state: present
     resource_definition:
       apiVersion: "{{ to_be_owned.apiVersion }}"
@@ -180,7 +180,7 @@ This file can be used as-is without user adjustments.
 
 - name: Patch resource with owner annotation
   when: to_be_owned.namespace is not defined or to_be_owned.namespace != owning_resource.namespace
-  k8s:
+  community.kubernetes.k8s:
     state: present
     resource_definition:
       apiVersion: "{{ to_be_owned.apiVersion }}"

doc/ansible/dev/testing_guide.md

@@ -218,7 +218,7 @@ We'll be adding the task to `roles/example/tasks/main.yml`, which should now loo
 ---
 # tasks file for exampleapp
 - name: create Example configmap
-  k8s:
+  community.kubernetes.k8s:
     definition:
       apiVersion: v1
       kind: ConfigMap
@@ -246,11 +246,11 @@ The file should now look like this:
   tasks:
     - debug: var=cm
       vars:
-        cm: '{{ lookup("k8s", api_version="v1", kind="ConfigMap", namespace=namespace, resource_name="test-data") }}'
+        cm: '{{ lookup("community.kubernetes.k8s", api_version="v1", kind="ConfigMap", namespace=namespace, resource_name="test-data") }}'
     - assert:
         that: cm.data.hello == 'world'
       vars:
-        cm: '{{ lookup("k8s", api_version="v1", kind="ConfigMap", namespace=namespace, resource_name="test-data") }}'
+        cm: '{{ lookup("community.kubernetes.k8s", api_version="v1", kind="ConfigMap", namespace=namespace, resource_name="test-data") }}'
 ```
 
 Now that we have a functional Operator, and an assertion of its behavior, we can verify that everything is working

doc/ansible/user-guide.md

@@ -201,7 +201,7 @@ Modify `roles/memcached/tasks/main.yml` to look like the following:
 ```yaml
 ---
 - name: start memcached
-  k8s:
+  community.kubernetes.k8s:
     definition:
       kind: Deployment
       apiVersion: apps/v1

internal/scaffold/ansible/dockerfilehybrid.go

@@ -79,7 +79,7 @@ RUN yum clean all && rm -rf /var/cache/yum/* \
  && yum remove -y gcc libffi-devel openssl-devel python36-devel \
  && yum clean all \
  && rm -rf /var/cache/yum \
- && ansible-galaxy collection install operator_sdk.util
+ && ansible-galaxy collection install operator_sdk.util community.kubernetes
 
 COPY build/_output/bin/[[.ProjectName]] ${OPERATOR}
 COPY bin /usr/local/bin

internal/scaffold/ansible/molecule_cluster_destroy.go

@@ -46,6 +46,8 @@ const moleculeClusterDestroyAnsibleTmpl = `---
   connection: local
   gather_facts: false
   no_log: "{{ molecule_no_log }}"
+  collections:
+    - community.kubernetes
 
   tasks:
     - name: Delete namespace

internal/scaffold/ansible/molecule_cluster_playbook.go

@@ -41,6 +41,8 @@ const moleculeClusterPlaybookAnsibleTmpl = `---
   hosts: localhost
   connection: local
   gather_facts: no
+  collections:
+    - community.kubernetes
 
   tasks:
     - name: Ensure operator image is set

internal/scaffold/ansible/molecule_cluster_prepare.go

@@ -46,6 +46,9 @@ const moleculeClusterPrepareAnsibleTmpl = `---
   connection: local
   gather_facts: false
   no_log: "{{ molecule_no_log }}"
+  collections:
+   - community.kubernetes
+
   vars:
     deploy_dir: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/deploy"
 

internal/scaffold/ansible/molecule_cluster_verify.go

@@ -46,6 +46,9 @@ const moleculeClusterVerifyAnsibleTmpl = `---
   hosts: localhost
   connection: local
   gather_facts: no
+  collections:
+    - community.kubernetes
+
   vars:
     custom_resource: "{{ lookup('template', '/'.join([deploy_dir, 'crds/[[.Resource.FullGroup]]_[[.Resource.Version]]_[[.Resource.LowerKind]]_cr.yaml'])) | from_yaml }}"
 

internal/scaffold/ansible/molecule_test_local_playbook.go

@@ -43,6 +43,9 @@ func (m *MoleculeTestLocalPlaybook) GetInput() (input.Input, error) {
 const moleculeTestLocalPlaybookAnsibleTmpl = `---
 - name: Build Operator in Kubernetes docker container
   hosts: k8s
+  collections:
+   - community.kubernetes
+
   vars:
     image: [[.Resource.FullGroup]]/[[.ProjectName]]:testing
 
@@ -64,6 +67,9 @@ const moleculeTestLocalPlaybookAnsibleTmpl = `---
 - name: Converge
   hosts: localhost
   connection: local
+  collections:
+   - community.kubernetes
+
   vars:
     image: [[.Resource.FullGroup]]/[[.ProjectName]]:testing
     operator_template: "{{ '/'.join([template_dir, 'operator.yaml.j2']) }}"

internal/scaffold/ansible/playbook.go

@@ -40,6 +40,9 @@ func (p *Playbook) GetInput() (input.Input, error) {
 const playbookTmpl = `---
 - hosts: localhost
   gather_facts: no
+  collections:
+    - community.kubernetes
+    - operator_sdk.util
 
   tasks:
     - import_role:

internal/scaffold/ansible/roles_meta_main.go

@@ -100,4 +100,6 @@ dependencies: []
   # List your role dependencies here, one per line. Be sure to remove the '[]' above,
   # if you add dependencies to this list.
 collections:
-- operator_sdk.util`
+- operator_sdk.util
+- community.kubernetes
+`

test/ansible-memcached/memfin/tasks/main.yml

@@ -1,5 +1,5 @@
 - name: delete configmap for test
-  k8s:
+  community.kubernetes.k8s:
     kind: ConfigMap
     api_version: v1
     name: deleteme

test/ansible-memcached/secret/tasks/main.yml

@@ -1,5 +1,5 @@
 - name: Create test service
-  k8s:
+  community.kubernetes.k8s:
     definition:
       kind: Service
       api_version: v1

test/ansible-memcached/tasks.yml

@@ -1,6 +1,6 @@
 ---
 - name: start memcached
-  k8s:
+  community.kubernetes.k8s:
     definition:
       kind: Deployment
       apiVersion: apps/v1
@@ -44,7 +44,7 @@
     status:
       test: "hello world"
 
-- k8s:
+- community.kubernetes.k8s:
     definition:
       kind: Secret
       apiVersion: v1
@@ -55,10 +55,10 @@
         test: aGVsbG8K
 - name: Get cluster api_groups
   set_fact:
-    api_groups: "{{ lookup('k8s', cluster_info='api_groups', kubeconfig=lookup('env', 'K8S_AUTH_KUBECONFIG')) }}"
+    api_groups: "{{ lookup('community.kubernetes.k8s', cluster_info='api_groups', kubeconfig=lookup('env', 'K8S_AUTH_KUBECONFIG')) }}"
 
 - name: create project if projects are available
-  k8s:
+  community.kubernetes.k8s:
     definition:
       apiVersion: project.openshift.io/v1
       kind: Project
@@ -67,7 +67,7 @@
   when: "'project.openshift.io' in api_groups"
 
 - name: Create ConfigMap to test blacklisted watches
-  k8s:
+  community.kubernetes.k8s:
     definition:
       kind: ConfigMap
       apiVersion: v1

test/ansible-memcached/verify.yml

@@ -3,10 +3,14 @@
 - name: Verify
   hosts: localhost
   connection: local
+  collections:
+    - community.kubernetes
+
   vars:
     ansible_python_interpreter: '{{ ansible_playbook_python }}'
     deploy_dir: "{{ lookup('env', 'MOLECULE_PROJECT_DIRECTORY') }}/deploy"
     custom_resource: "{{ lookup('file', '/'.join([deploy_dir, 'crds/ansible.example.com_v1alpha1_memcached_cr.yaml'])) | from_yaml }}"
+
   tasks:
     - block:
       - name: debug memcached lookup

test/ansible/build/Dockerfile

@@ -11,5 +11,5 @@ COPY fixture_collection/ /tmp/fixture_collection/
 USER root
 RUN chmod -R ug+rwx /tmp/fixture_collection
 USER 1001
-RUN ansible-galaxy collection build /tmp/fixture_collection/ --output-path /tmp/fixture_collection/
-RUN ansible-galaxy collection install /tmp/fixture_collection/operator_sdk-test_fixtures-0.0.0.tar.gz
+RUN ansible-galaxy collection build /tmp/fixture_collection/ --output-path /tmp/fixture_collection/ \
+ && ansible-galaxy collection install /tmp/fixture_collection/operator_sdk-test_fixtures-0.0.0.tar.gz

test/ansible/fixture_collection/roles/dummy/tasks/main.yml

@@ -1,6 +1,6 @@
 ---
 - name: Create ConfigMap
-  k8s:
+  community.kubernetes.k8s:
     definition:
       kind: ConfigMap
       apiVersion: v1

test/ansible/molecule/cluster/playbook.yml

@@ -3,6 +3,8 @@
   hosts: localhost
   connection: local
   gather_facts: no
+  collections:
+    - community.kubernetes
 
   tasks:
     - name: Ensure operator image is set

test/ansible/molecule/cluster/verify.yml

@@ -3,7 +3,10 @@
   hosts: localhost
   connection: local
   gather_facts: no
+  collections:
+    - community.kubernetes
   tasks:
-    - import_tasks: tasks/inventory_test.yml
-    - import_tasks: tasks/servicemonitor_test.yml
-    - import_tasks: tasks/collections_test.yml
+    - name: Import all test files from tasks/
+      include_tasks: '{{ item }}'
+      with_fileglob:
+        - tasks/*_test.yml

test/ansible/molecule/test-local/playbook.yml

@@ -1,8 +1,12 @@
 ---
 - name: Build Operator in Kubernetes docker container
   hosts: k8s
+  collections:
+    - community.kubernetes
+
   vars:
     image: test.example.com/ansible:testing
+
   tasks:
     # using command so we don't need to install any dependencies
     - name: Get existing image hash
@@ -21,6 +25,9 @@
 - name: Converge
   hosts: localhost
   connection: local
+  collections:
+    - community.kubernetes
+
   vars:
     image: test.example.com/ansible:testing
     operator_template: "{{ '/'.join([template_dir, 'operator.yaml.j2']) }}"

test/ansible/roles/inventory/tasks/main.yml

@@ -1,7 +1,7 @@
 ---
 - when: sentinel | test
   block:
-  - k8s:
+  - community.kubernetes.k8s:
       definition:
         apiVersion: v1
         kind: ConfigMap
@@ -10,4 +10,4 @@
           namespace: '{{ meta.namespace }}'
         data:
           sentinel: '{{ sentinel }}'
-          groups: '{{ groups | to_nice_yaml }}' 
+          groups: '{{ groups | to_nice_yaml }}'