Devops/2337 deploy develop on pr merge (#2401)

* added Action to kick off deploys to develop on merges to develop, with additional job of running e2e tests on deployed environment

* fixed wrong parameter name in workflow

* don't need to run tests in circleCI before deploy (should already have been run in PR to develop) and temporarily change branch tracked for develop env

* update deploy script to add in DJANGO_DEBUG and CYPRESS_TOKEN to env for develop deploys

* updated to track develop branch before PR to develop. Added to documentation

* added empty line at and of file

* added empty line at and of file

* moved base and api url prefixes to env vars

---------

Co-authored-by: George Hudson <[email protected]>
Co-authored-by: Andrew <[email protected]>

Author: 3 people

.circleci/base_config.yml

@@ -16,15 +16,18 @@ executors:
       image: ubuntu-2204:2022.10.1
 
 parameters:
+  develop_branch_deploy:
+    type: boolean
+    default: false
   run_dev_deployment:
     type: boolean
     default: false
+  run_nightly_owasp_scan:
+    type: boolean
+    default: false
   run_owasp_scan:
     type: boolean
     default: false
   target_env:
     type: string
     default: ''
-  run_nightly_owasp_scan:
-    type: boolean
-    default: false

.circleci/build-and-test/workflows.yml

@@ -3,6 +3,7 @@
     unless:
       or:
         - << pipeline.parameters.run_dev_deployment >>
+        - << pipeline.parameters.develop_branch_deploy >>
         - << pipeline.parameters.run_owasp_scan >>
     jobs:
       - secrets-check

.circleci/config.yml

@@ -13,18 +13,21 @@ orbs:
 
 # parameters from github actions
 parameters:
+  develop_branch_deploy:
+    type: boolean
+    default: false
   run_dev_deployment:
     type: boolean
     default: false
+  run_nightly_owasp_scan:
+    type: boolean
+    default: false
   run_owasp_scan:
     type: boolean
     default: false
   target_env:
     type: string
     default: ''
-  run_nightly_owasp_scan:
-    type: boolean
-    default: false
 
 jobs:
   setup:

.circleci/deployment/jobs.yml

@@ -32,6 +32,23 @@
           cf-space: tanf-staging
           cf-username: CF_USERNAME_STAGING
 
+  test-deployment-e2e:
+    executor: machine-executor
+    working_directory: ~/tdp-apps
+    steps:
+      - checkout
+      - install-nodejs-machine
+      - disable-npm-audit
+      - install-nodejs-packages:
+          app-dir: tdrs-frontend
+      - run:
+          name: Run Cypress e2e tests
+          command: cd tdrs-frontend; npm run test:e2e-develop
+      - store_artifacts:
+          path: tdrs-frontend/cypress/screenshots/
+      - store_artifacts:
+          path: tdrs-frontend/cypress/videos/
+
   deploy-production:
     executor: docker-executor
     working_directory: ~/tdp-deploy

.circleci/deployment/workflows.yml

@@ -42,6 +42,13 @@
             branches:
               only:
                 - develop
+      - test-deployment-e2e:
+          requires:
+            - deploy-develop
+          filters:
+            branches:
+              only:
+                - develop
 
   production-deployment:
     unless: << pipeline.parameters.run_dev_deployment >>

.github/workflows/deploy-develop-on-merge.yml

@@ -0,0 +1,48 @@
+###########################################################################
+# GitHub Action Workflow
+# On push to the develop branch (which should only be done through PR)
+# in GitHub this action will trigger a deploy job within CircleCI for the
+# deployment and e2e testing of the develop environment.
+#
+# Step 0: Checkout latest commit on develop
+#
+# Step 1: Makes a request to the V2 CircleCI API to initiate the project,
+#         which will filter based upon the branch to initiate the
+#         workflow/jobs listed here:
+#           staging-deployment:[
+#             deploy-project-updates-site,
+#             deploy-infrastructure-develop,
+#             deploy-develop,
+#             test-deployment-e2e
+#           ]
+#
+#         Leverages the open source GitHub Action:
+#           https://github.com/promiseofcake/circleci-trigger-action
+###########################################################################
+name: Deploy Develop on PR Merge
+on:
+  push:
+    branches:
+    - develop
+    paths_ignore:
+    - 'docs/**'
+    - '**.md'
+    - '**.txt'
+    - '.gitattributes'
+    - '.gitignore'
+    - 'LICENSE'
+jobs:
+  merge_deployment:
+    if: github.ref == 'refs/heads/develop'
+    runs-on: ubuntu-latest
+    name: Initiate deploy job in CircleCI
+    steps:
+    - uses: actions/checkout@v2
+    - name: Circle CI Deployment Trigger
+      id: curl-circle-ci
+      uses: promiseofcake/circleci-trigger-action@v1
+      with:
+        user-token: ${{ secrets.CIRCLE_CI_V2_TOKEN }}
+        project-slug: raft-tech/TANF-app
+        branch: ${{ github.ref }}
+        payload: '{"develop_branch_deploy": true, "target_env": "develop"}'

docs/Technical-Documentation/README.md

@@ -12,6 +12,7 @@ This directory contains system and architecture documentation including diagrams
 - [buildpack-changelog.md](./buildpack-changelog.md) : A running log of updates to our Cloud.gov buildpacks in use.
 - [circle-ci-audit-template.md](./circle-ci-audit-template.md) : This is a checklist document used during audits of our continuous intergration pipeline tool.
 - [circle-ci.md](./circle-ci.md) : Overview of our CI/CD platform jobs.
+- [cypress-integration-tests.md](./cypress-integration-tests.md) : Shows how we use Cypress to manage our end to end integration testing.
 - [data-file-downloads.md](./data-file-downloads.md) : Provides an architecture-level view of data file storage and downloading.
 - [django-admin-logging.md](./django-admin-logging.md) : Outlines sections of the Django Administrator Console and details what should be logged.
 - [jwt-key-rotation.md](./jwt-key-rotation.md) : Describes the process for rotating JWT keys in Login.gov.

docs/Technical-Documentation/circle-ci.md

@@ -101,3 +101,18 @@ These all have defaults set in their respective settings modules, but may be ove
         * Bind the backend application to the S3 and RDS services in Cloud.gov
         * Run `/scripts/set-backend-env-vars.sh` (detailed above)
         * Restage the application to make environment variable and bound services live.
+
+## CI/CD Pipeline
+
+The Frontend and Backend deploy Workflows are triggered automatically on pushes to the `develop` branch and on labeled PRs.
+* GitHub Actions are used to notify CircleCI to run if criteria are met and pass along variables to the workflows as needed.
+    * Documentation for what each Action does can be found in each actions workflow action-name.yml file.
+    * These files are located in [.github/workflows](../../.github/workflows/).
+* merges to the `develop` branch will deploy to the develop environment.
+    * on successful develop deploys, end to end integration tests will then be run against that environment.
+    * for details, refer to the [Cypress Integration Tests](./cypress-integration-tests.md) documentation.
+* PRs given the explicit label `Deploy with CircleCI` will deploy to the tanf-dev environment selected
+    * `Deploy with CircleCI` is the prefix part of the label which triggers the build. It needs the environment added as a suffix
+    * To select the environment, add the name after a hyphen following the `Deploy with CircleCI` prefix
+    * e.g. `Deploy with CircleCI-raft` will deploy your branch build to the tanf-dev Cloud Foundry space, tdp-raft environment
+        * tdp-frontend-raft & tdp-backend-raft

docs/Technical-Documentation/cypress-integration-tests.md

@@ -4,6 +4,10 @@
 
 [Cypress](https://cypress.io) is a browser-automation testing suite that we use for end-to-end tests. See [ADR019 - Integration Tests](./Architecture-Decision-Record/019-integration-tests.md) for some additional background on testing goals and decisions.
 
+## CI/CD Pipeline
+
+All tests added into the `tdrs-frontend/cypress/e2e/` folder will be run against the newly deployed develop environment as part of our pipeline to help ensure site reliability and that no bugs have been introduced. These are run against the actual deployed environment and so might catch issues connecting with real backend services instead of having everything local.
+
 ## Running tests
 
 1. Have both the backend and frontend running in separate terminal processes, the app needs to be reachable and usable at `localhost:3000` when testing locally

scripts/deploy-backend.sh

@@ -42,7 +42,9 @@ set_cf_envs()
   "AV_SCAN_URL"
   "BASE_URL"
   "CLAMAV_NEEDED"
+  "CYPRESS_TOKEN"
   "DJANGO_CONFIGURATION"
+  "DJANGO_DEBUG"
   "DJANGO_SECRET_KEY"
   "DJANGO_SETTINGS_MODULE"
   "DJANGO_SU_NAME"
@@ -171,6 +173,8 @@ elif [ "$CF_SPACE" = "tanf-staging" ]; then
   DJANGO_CONFIGURATION="Staging"
 else
   DJANGO_CONFIGURATION="Development"
+  DJANGO_DEBUG="Yes"
+  CYPRESS_TOKEN=$CYPRESS_TOKEN
 fi
 
 if [ "$DEPLOY_STRATEGY" = "rolling" ] ; then

tdrs-frontend/package.json

@@ -39,6 +39,7 @@
     "test:accessibility": "concurrently -k -s first 'REACT_APP_PA11Y_TEST=true npm run start:ci' 'wait-on http://localhost:3000/ && npm run pa11y-ci'",
     "test:e2e": "cypress open",
     "test:e2e-ci": "cypress run --headless -b chrome --env cypressToken=${CYPRESS_TOKEN}",
+    "test:e2e-develop": "cypress run --headless -b chrome --config baseUrl=https://${BASE_URL_PREFIX}-develop.app.cloud.gov --env cypressToken=${CYPRESS_TOKEN},apiUrl=https://${API_URL_PREFIX}-develop.app.cloud.gov/v1",
     "eject": "react-scripts eject",
     "lint": "eslint src/ && echo 'Lint complete.'",
     "pa11y-ci": "pa11y-ci --config .pa11yci.json"