Kubernetes service (#5)

* Add kubernetes service tasks

* Update README.md with appropriate screenshot and tasks

README.md

@@ -10,3 +10,8 @@ Catalog of Tasks usable in [Continuous Delivery Tekton Pipelines](https://cloud.
 
 - **containerize-task**: This task is building and pushing an image to [IBM Cloud Container Registry](https://cloud.ibm.com/docs/services/Registry?topic=registry-getting-started). This taks is relying on [Buildkit](https://github.com/moby/buildkit) to perform the build of the image. [Documentation is here](./container-registry/README.md)
 - **vulnerability-advisor-task**: This task is verifying that a [Vulnerability Advisor scan](https://cloud.ibm.com/docs/services/Registry?topic=va-va_index) has been made for the image and process the outcome of the scan. [Documentation is here](./container-registry/README.md)
+
+## IBM Cloud Kubernetes Service related tasks
+
+- **fetch-iks-cluster-config**: This task is fetching the configuration of a [IBM Cloud Kubernetes Service cluster](https://cloud.ibm.com/docs/containers?topic=containers-getting-started) that is required to perform `kubectl` commands. [Documentation is here](./kubernetes-service/README.md)
+- **kubernetes-contextual-execution**: This task is executing bash snippet/script in the context of a Kubernetes cluster configuration. [Documentation is here](./kubernetes-service/README.md)

kubernetes-service/README.md

@@ -0,0 +1,98 @@
+# Kubernetes-Service related tasks
+
+- **fetch-iks-cluster-config**: This task is fetching the configuration of a [IBM Cloud Kubernetes Service cluster](https://cloud.ibm.com/docs/containers?topic=containers-getting-started) that is required to perform `kubectl` commands.
+- **kubernetes-contextual-execution**: This task is executing bash snippet/script in the context of a Kubernetes cluster configuration.
+
+**WARNING: These tasks needs to run on Kubernetes cluster with minimal version 1.16. If you are using your own Delivery Pipeline Private Worker to run your tekton pipeline(s), ensure your cluster is updated to this version at least.**
+
+## Install the Tasks
+- Add a github integration in your toolchain to the repository containing the task (https://github.com/open-toolchain/tekton-catalog)
+- Add that github integration to the Definitions tab of your Continuous Delivery tekton pipeline, with the Path set to `kubernetes-service`
+
+## Fetch IKS Cluster Configuration helper task
+
+### Inputs
+
+#### Context - ConfigMap/Secret
+
+  The task expects the following kubernetes resources to be defined:
+
+* **Secret cd-secret**
+
+  Secret containing:
+  * **API_KEY**: An IBM Cloud Api Key use to access to the IBM Cloud Container registry service (https://cloud.ibm.com/iam/apikeys)
+
+  See [sample TriggerTemplate](./sample/listener-kubernetes-service.yaml) on how to create the secret using `resourcetemplates` in a `TriggerTemplate`
+
+#### Resources
+
+* **cluster**: The Cluster PipelineResource that will be updated as output of this task. Only the name property is used to identify the cluster name.
+
+#### Parameters
+
+* **task-pvc**: the output pvc - this is the name of the PVC that is mounted for the execution of the task
+* **resourceGroup**: (optional) target resource group (name or id) for the ibmcloud login operation.
+* **clusterRegion**: (optional) the ibmcloud region hosting the target cluster. If not specified, it will use the toolchain region as a default.
+* **clusterPipelineResourcesDirectoryFallback**: (optional) that will be used as a fallback mechanism to store the kubeconfig file for the target cluster (expressed by the inputs)
+
+### Outputs
+
+#### Resources
+
+* **cluster**: The Cluster PipelineResource that will be updated as output of this task.
+
+## Kubernetes Contextual Execution helper task
+
+### Inputs
+
+#### Context - ConfigMap/Secret
+
+  The task expects the following kubernetes resources to be defined:
+
+* **Secret cd-secret**
+
+  Secret containing:
+  * **API_KEY**: An IBM Cloud Api Key use to access to the IBM Cloud Container registry service (https://cloud.ibm.com/iam/apikeys)
+
+  See [sample TriggerTemplate](./sample/listener-kubernetes-service.yaml) on how to create the secret using `resourcetemplates` in a `TriggerTemplate`
+
+#### Resources
+
+* **cluster**: The Cluster PipelineResource that corresponds to the kubernetes cluster target for the kubectl command execution.
+
+#### Parameters
+
+* **task-pvc**: the output pvc - this is the name of the PVC that is mounted for the execution of the task
+* **task-pvc-mountpath**: the mountpath for the pvc (default to `/artifacts`)
+* **clusterPipelineResourcesDirectory**: directory in which the kubeconfig file(s) for clusterPipelineResources are available (default to `/workspace` but this may need to be a combination of `task-pvc-mountpath` and `fetch-iks-cluster-config#clusterPipelineResourcesDirectoryFallback` if cluster pipeline resource update is not made by the `fetch-iks-cluster-config` task - ie using the fallback mechanism of kubeconfig copy to the pipelinerun pvc)
+* **script**: the bash snippet to execute within the context of the kubernetes configuration (default to `kubectl version`)
+
+# Usage
+The `sample` sub-directory contains an EventListener definition that you can include in your tekton pipeline configuration to run an example usage of the `fetch-iks-cluster-config` and `kubernetes-contextual-execution` tasks.
+
+1) Create a toolchain (or update a toolchain) to include:
+
+   - the git repository that you want to clone, which can be private
+   - the repository containing this tekton task
+   - a tekton pipeline definition
+
+   ![Toolchain overview](./sample/kubernetes-service-sample-toolchain-overview.png)
+
+2) Add the definitions:
+
+   - for the tasks and the sample (`kubernetes-service` and `kubernetes-service/sample` paths)
+
+   ![Tekton pipeline definitions](./sample/kubernetes-service-sample-tekton-pipeline-definitions.png)
+
+3) Add the environment properties:
+
+   - `apikey` to provide an API key used for the ibmcloud login/access
+   - `clusterName` to indicate the name of the IKS cluster that you want to target
+
+   ![Tekton pipeline environment properties](./sample/kubernetes-service-sample-tekton-pipeline-environment-properties.png)
+
+4) Create a manual trigger to start the sample listener
+
+   ![Tekton pipeline sample trigger](./sample/kubernetes-service-sample-tekton-pipeline-sample-triggers.png)
+
+5) Run the pipeline

kubernetes-service/sample/listener-kubernetes-service.yaml

@@ -0,0 +1,67 @@
+apiVersion: tekton.dev/v1alpha1
+kind: TriggerTemplate
+metadata:
+  name: trigger-template-kubernetes-service
+spec:
+  params:
+    - name: apikey
+      description: the ibmcloud api key
+    - name: clusterName
+      description: the name of the cluster to target
+  resourcetemplates:
+    - apiVersion: v1
+      kind: Secret
+      metadata:
+        name: cd-secret
+      type: Opaque
+      stringData:
+        API_KEY: $(params.apikey)
+    - apiVersion: tekton.dev/v1alpha1
+      kind: PipelineResource
+      metadata:
+        name: the-cluster
+      spec:
+        type: cluster
+        params:
+          - name: name
+            value: $(params.clusterName)
+          - name: username
+            value: norealvalueneeded
+          - name: url
+            value: https://no.real.value.needed
+          - name: token
+            value: norealvalueneeded
+          - name: cadata
+            value: norealvalueneeded
+    - apiVersion: tekton.dev/v1alpha1
+      kind: PipelineRun
+      metadata:
+        name: pipelinerun-$(uid)
+      spec:
+        pipelineRef:
+          name: pipeline-kubernetes-service
+        resources:
+          - name: target-cluster
+            resourceRef:
+              name: the-cluster
+        params:
+          - name: pipeline-pvc
+            value: pipelinerun-$(uid)-pvc
+---
+apiVersion: tekton.dev/v1alpha1
+kind: TriggerBinding
+metadata:
+  name: trigger-binding-kubernetes-service
+spec:
+  params:
+---
+apiVersion: tekton.dev/v1alpha1
+kind: EventListener
+metadata:
+  name: event-listener-kubernetes-service
+spec:
+  triggers:
+    - binding:
+        name: trigger-binding-kubernetes-service
+      template:
+        name: trigger-template-kubernetes-service

kubernetes-service/sample/pipeline-kubernetes-service.yaml

@@ -0,0 +1,51 @@
+apiVersion: tekton.dev/v1alpha1
+kind: Pipeline
+metadata:
+  name: pipeline-kubernetes-service
+spec:
+  params:
+    - name: pipeline-pvc
+      description: the pipeline pvc name
+  resources:
+  - name: target-cluster
+    type: cluster
+  tasks:
+    - name: pipeline-0-setup-task
+      taskRef:
+        name: fetch-iks-cluster-config
+      params:
+        - name: task-pvc
+          value: $(params.pipeline-pvc)
+        - name: clusterPipelineResourcesDirectoryFallback
+          value: .tekton-clusters
+      resources:          
+        inputs:
+          - name: cluster
+            resource: target-cluster
+        outputs:
+          - name: cluster
+            resource: target-cluster
+    - name: pipeline-1-kubectl-task
+      taskRef:
+        name: kubernetes-contextual-execution
+      params:
+        - name: task-pvc
+          value: $(params.pipeline-pvc)
+        - name: task-pvc-mountpath
+          value: /pipelinerun
+        - name: clusterPipelineResourcesDirectory
+          value: /pipelinerun/.tekton-clusters
+        - name: script
+          value: |
+            echo "** Here is the kubectl version:"
+            kubectl version
+            echo "** Here is the kubectl cluster-info:"
+            kubectl cluster-info
+            echo "** Here are the kubectl namespaces:"
+            kubectl get namespaces
+      resources:          
+        inputs:
+          - name: cluster
+            resource: target-cluster
+            from:
+              - pipeline-0-setup-task

kubernetes-service/task-fetch-iks-cluster-config.yaml

@@ -0,0 +1,126 @@
+apiVersion: tekton.dev/v1alpha1
+kind: Task
+metadata:
+  name: fetch-iks-cluster-config
+spec:
+  inputs:
+    params:
+      - name: task-pvc
+        description: the task pvc - this is the volume where the files (Dockerfile etc..) are expected to be
+      - name: ibmcloudApi
+        description: the ibmcloud api
+        default: https://cloud.ibm.com
+      - name: continuous-delivery-context-secret
+        description: name of the configmap containing the continuous delivery pipeline context secrets
+        default: cd-secret
+      - name: resourceGroup
+        description: target resource group (name or id) for the ibmcloud login operation
+        default: ''
+      - name: clusterRegion
+        description: (optional) the ibmcloud region hosting the cluster (if none is found it will default to the toolchain region)
+        default: ''
+      - name: kubeAPIServerAccessible
+        description: |
+          indicates if the kubeAPIServer is exposed which is not the case for IBM Cloud Public Shared Workers (Calico network policy).
+          If 'true', the task is trying to update the Cluster Pipeline Resources definition with the appropriate informations
+          When 'false', the fallback mechanism (copy file(s)) is used.
+        default: 'false'
+      - name: clusterPipelineResourcesDirectoryFallback
+        description: directory in the task-pvc that will be used as a fallback mechanism to store the kubeconfig file
+        default: .tekton-cluster-pipeline-resources
+    resources:
+      # Will be set as optional when tekton version support it
+      - name: cluster
+        type: cluster
+  outputs:
+    resources:
+      # Will be set as optional when tekton version support it
+      - name: cluster
+        type: cluster    
+  steps:
+    - name: setup
+      image: ibmcom/pipeline-base-image
+      workingDir: /artifacts
+      env:
+        - name: API_KEY
+          valueFrom:
+            secretKeyRef:
+              name: $(inputs.params.continuous-delivery-context-secret)
+              key: API_KEY
+        - name: NAMESPACE
+          valueFrom:
+            fieldRef:
+              fieldPath: metadata.namespace
+      command: ["/bin/bash", "-c"]
+      args:
+        - |
+          set -e -o pipefail
+          ##########################################################################
+          # Setting HOME explicitly to have ibmcloud plugins available
+          # doing the export rather than env definition is a workaround
+          # until https://github.com/tektoncd/pipeline/issues/1836 is fixed
+          export HOME="/root"
+          ##########################################################################
+          ibmcloud config --check-version false
+          if [ "$(inputs.params.clusterRegion)" ]; then
+            TARGET_REGION=$(inputs.params.clusterRegion)
+          else 
+            TARGET_REGION=$(jq -r '.region_id' /cd-config/toolchain.json | awk -F: '{print $3}')
+          fi          
+          ibmcloud login -a $(inputs.params.ibmcloudApi) -r $TARGET_REGION --apikey $API_KEY
+          # Look for the cluster
+          if ibmcloud ks cluster get $(inputs.resources.cluster.name); then
+            KUBECONFIG_FILE=$(ibmcloud ks cluster config $(inputs.resources.cluster.name) --export -s | awk -F= '{print $2}')
+          else 
+            echo "Cluster $(inputs.resources.cluster.name) not found. Accessible clusters are:"
+            ibmcloud ks clusters
+            exit 1
+          fi
+
+          # PipelineResource Cluster are managed appropriately as input resources
+          # but not as output resources, attempting to update the PipelineResource Cluster
+          # that has been given (using kubectl patch commands)
+          # Verifying the access to pipeline resources
+          updateClusterPipelineResource="false"
+          if [ "$(inputs.params.kubeAPIServerAccessible)" == "true" ]; then
+            if kubectl auth can-i get pipelineresources.tekton.dev > /dev/null 2>&1; then
+              updateClusterPipelineResource="true"
+            fi
+          fi
+          if [ "$updateClusterPipelineResource" == "true" ]; then
+            # TODO Need to update the Cluster Pipeline Resource with this values
+            # This can only be done when TriggerTemplate#resourcetemplates will accept
+            # Role and Binding K8S resources definition
+            echo "TODO - Processing $KUBECONFIG_FILE"
+            K8S_USER_NAME=$(yq r $KUBECONFIG_FILE users[0].name)
+            K8S_USER_TOKEN=$(yq r $KUBECONFIG_FILE users[0].user.auth-provider.config.id-token)
+            K8S_CLUSTER_URL=$(yq r $KUBECONFIG_FILE clusters[0].cluster.server)
+            CA_PEM_FILE="$(dirname $KUBECONFIG_FILE)/$(yq r $KUBECONFIG_FILE clusters[0].cluster.certificate-authority)"
+            K8S_CLUSTER_CADATA=$(base64 -w0 $CA_PEM_FILE)
+            echo "K8S_USER_NAME=$K8S_USER_NAME"
+            #echo "K8S_USER_TOKEN=$K8S_USER_TOKEN"
+            echo "K8S_CLUSTER_URL=$K8S_CLUSTER_URL"
+            #echo "K8S_CLUSTER_CADATA=$K8S_CLUSTER_CADATA"
+          else
+            echo "Access to pipelineresources.tekton.dev is required to update Cluser Pipeline Resource accordingly"
+            echo "Falling back to copy of $(basename $KUBECONFIG_FILE) to the pipeline run volume"
+            mkdir -p /artifacts/$(inputs.params.clusterPipelineResourcesDirectoryFallback)/$(inputs.resources.cluster.name)
+            cp $(dirname $KUBECONFIG_FILE)/$(yq r $KUBECONFIG_FILE clusters[0].cluster.certificate-authority) /artifacts/$(inputs.params.clusterPipelineResourcesDirectoryFallback)/$(inputs.resources.cluster.name)
+            cp $KUBECONFIG_FILE /artifacts/$(inputs.params.clusterPipelineResourcesDirectoryFallback)/$(inputs.resources.cluster.name)/kubeconfig
+            echo "Kubeconfig file(s) copied to /artifacts/$(inputs.params.clusterPipelineResourcesDirectoryFallback)/$(inputs.resources.cluster.name)"
+          fi
+      volumeMounts:
+        - mountPath: /artifacts
+          name: task-volume
+        - mountPath: /cd-config
+          name: cd-config-volume
+  volumes:
+    - name: task-volume
+      persistentVolumeClaim:
+        claimName: $(inputs.params.task-pvc)
+    - name: cd-config-volume
+      configMap:
+        name: toolchain
+        items:
+        - key: toolchain.json
+          path: toolchain.json