From 039890925eb4cc3978600979d77eb2c50074aa69 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Wed, 12 Feb 2025 11:13:23 -0800 Subject: [PATCH 01/17] Remove adhoc repo refs --- README.md | 451 ++++++++++++++++++++++++++++-------------------------- 1 file changed, 232 insertions(+), 219 deletions(-) diff --git a/README.md b/README.md index 89c41db581..ec048c79a4 100644 --- a/README.md +++ b/README.md @@ -5,18 +5,12 @@ Welcome to the home of the OHS TTAHUB. - -
HHSAd Hoc
[![HHS](https://circleci.com/gh/HHS/Head-Start-TTADP.svg?style=shield)](https://app.circleci.com/pipelines/github/HHS/Head-Start-TTADP) - - -[![adhocteam](https://circleci.com/gh/adhocteam/Head-Start-TTADP.svg?style=shield)](https://app.circleci.com/pipelines/github/adhocteam/Head-Start-TTADP) -
@@ -29,7 +23,7 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr ### Set up -*Warning* when using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of +_Warning_ when using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of those services are already running on your machine. #### Docker @@ -39,8 +33,8 @@ those services are already running on your machine. 3. Copy `.env.example` to `.env`. 4. Change the `AUTH_CLIENT_ID` and `AUTH_CLIENT_SECRET` variables to to values found in the team Keybase account. If you don't have access to Keybase, please ask in the acf-head-start-eng slack channel for access. 5. Optionally, set `CURRENT_USER` to your current user's uid:gid. This will cause files created by docker compose to be owned by your user instead of root. -6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: ```"The chromium binary is not available for arm64"```. See the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. -7. Run `yarn docker:start` to start the application. The [frontend][frontend] will be available on `localhost:3000` and the [backend][backend] will run on `localhost:8080`, [API documentation][API documentation] will run on `localhost:5003`, and [minio][minio] will run on `localhost:9000`. +6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`. See the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. +7. Run `yarn docker:start` to start the application. The [frontend][frontend] will be available on `localhost:3000` and the [backend][backend] will run on `localhost:8080`, [API documentation][API documentation] will run on `localhost:5003`, and [minio][minio] will run on `localhost:9000`. 8. Run `yarn docker:stop` to stop the servers and remove the docker containers. The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. @@ -52,6 +46,7 @@ Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve docume Make sure you have access to all the necessary spaces on Cloud.gov On a Mac + 1. Login to cloud.gov: `cf login -a api.fr.cloud.gov --sso`. 2. Download latest data: `bash ./bin/latest_backup.sh -d` (file will be placed in current directory). 3. Ensure you have `psql` (if not `brew install libpq`). @@ -60,9 +55,10 @@ On a Mac 6. Load data: `psql postgresql://username:password@127.0.0.1:5432/postgres < ./bounce.sql && psql postgresql://username:password@127.0.0.1:5432/ttasmarthub < db.sql` (Where username:password are replaced with credentials from .env and db.sql is the file you downloaded and unzipped). 7. Migrate data: `yarn docker:db:migrate` 8. Edit .env and change CURRENT_USER_ID= from 1 to the ID of a production user -9. Restart docker +9. Restart docker bounce.sql + ```sh select pg_terminate_backend(pid) from pg_stat_activity where datname='ttasmarthub'; drop database ttasmarthub; @@ -73,8 +69,9 @@ On Windows TBD #### Apple Silicon & Chromium + On a Mac with Apple Silicon, puppeteer install fails with the message: -```"The chromium binary is not available for arm64"``` +`"The chromium binary is not available for arm64"` See [docker-compose.override.yml](docker-compose.override.yml) and uncomment the relevant lines to skip downloading chromium and use the host's binary instead. @@ -87,9 +84,9 @@ export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true export PUPPETEER_EXECUTABLE_PATH=`which chromium` ``` -On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: +On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: -`brew install docker-compose` +`brew install docker-compose` #### Local build @@ -99,35 +96,37 @@ You must also install and run minio locally to use the file upload functionality #### Precommit hooks -Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](https://github.com/adhocteam/Head-Start-TTADP/tree/main/terraform#set-up). +Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](https://github.com/HHS/Head-Start-TTADP/tree/main/terraform#README.md). If you are not using your own custom pre-commit hooks: - **start from repo root directory** - **make the pre-commit file executable** -chmod 755 .githooks/pre-commit + chmod 755 .githooks/pre-commit - **change your default hooks directory to `.githooks`.** -git config core.hooksPath .githooks + git config core.hooksPath .githooks If you are already using git hooks, add the .githooks/pre-commit contents to your hooks directory or current pre-commit hook. Remember to make the file executable. ### Building Tests #### Helpful notes on writing (backend) tests + It's important that our tests fully clean up after themselves if they interact with the database. This way, tests do not conflict when run on the CI and remain as deterministic as possible.The best way to do this is to run them locally in an isolated environment and confirm that they are sanitary. With that in mind, there a few "gotchas" to remember to help write sanitary tests. -- ```Grant.destroy``` needs to run with ```individualHooks: true``` or the related GrantNumberLink model prevents delete. Additionally, the hooks on destroy also update the materialized view (GrantRelationshipToActive). -- When you call ```Model.destroy``` you should be adding ```individualHooks: true``` to the Sequelize options. Often this is required for proper cleanup. There may be times when this is undesirable; this should be indicated with a comment. -- Be aware of paranoid models. For those models: force: true gets around the soft delete. If they are already soft-deleted though, you need to remove the default scopes paranoid: true does it, as well as Model.unscoped() -- There are excellent helpers for creating and destroying common Model mocks in ```testUtils.js```. Be aware that they take a scorched earth approach to cleanup. For example, when debugging a flaky test, it was discovered that ```destroyReport``` was removing a commonly used region. + +- `Grant.destroy` needs to run with `individualHooks: true` or the related GrantNumberLink model prevents delete. Additionally, the hooks on destroy also update the materialized view (GrantRelationshipToActive). +- When you call `Model.destroy` you should be adding `individualHooks: true` to the Sequelize options. Often this is required for proper cleanup. There may be times when this is undesirable; this should be indicated with a comment. +- Be aware of paranoid models. For those models: force: true gets around the soft delete. If they are already soft-deleted though, you need to remove the default scopes paranoid: true does it, as well as Model.unscoped() +- There are excellent helpers for creating and destroying common Model mocks in `testUtils.js`. Be aware that they take a scorched earth approach to cleanup. For example, when debugging a flaky test, it was discovered that `destroyReport` was removing a commonly used region. - The next section details additional tools, found in `src/lib/programmaticTransaction.ts`, which make maintaining a clean database state when writing tests a breeze. #### Database State Management in Tests -The guidance is on using the `captureSnapshot` and `rollbackToSnapshot` functions from `src/lib/programmaticTransaction.ts` to manage database state during automated testing with Jest. These functions ensure that each test is executed in a clean state, preventing tests from affecting each other and improving test reliability. +The guidance is on using the `captureSnapshot` and `rollbackToSnapshot` functions from `src/lib/programmaticTransaction.ts` to manage database state during automated testing with Jest. These functions ensure that each test is executed in a clean state, preventing tests from affecting each other and improving test reliability. ##### Functions Overview @@ -141,7 +140,7 @@ The guidance is on using the `captureSnapshot` and `rollbackToSnapshot` function In this example, `captureSnapshot` and `rollbackToSnapshot` are used at the Jest suite level to manage database states before and after all tests run. This is useful when tests are not independent or when setup/teardown for each test would be too costly. ```javascript -describe('Database State Management', () => { +describe("Database State Management", () => { let snapshot; beforeAll(async () => { @@ -154,11 +153,11 @@ describe('Database State Management', () => { await transactionModule.rollbackToSnapshot(snapshot); }); - it('Test Case 1', async () => { + it("Test Case 1", async () => { // Test actions that modify the database }); - it('Test Case 2', async () => { + it("Test Case 2", async () => { // Further test actions that modify the database }); }); @@ -169,14 +168,14 @@ describe('Database State Management', () => { This approach uses `captureSnapshot` and `rollbackToSnapshot` at the start and end of each individual test. It is most effective when tests are meant to run independently, ensuring no residual data affects subsequent tests. ```javascript -describe('Individual Test Isolation', () => { - it('Test Case 1', async () => { +describe("Individual Test Isolation", () => { + it("Test Case 1", async () => { const snapshot = await transactionModule.captureSnapshot(); // Actions modifying the database await transactionModule.rollbackToSnapshot(snapshot); }); - it('Test Case 2', async () => { + it("Test Case 2", async () => { const snapshot = await transactionModule.captureSnapshot(); // More actions modifying the database await transactionModule.rollbackToSnapshot(snapshot); @@ -204,57 +203,59 @@ To run eslint run `yarn lint:all` or `yarn lint:fix:all` to have eslint attempt You may run into some issues running the docker commands on Windows: - * If you run into `Permission Denied` errors see [this issue](https://github.com/docker/for-win/issues/3385#issuecomment-501931980) - * You can try to speed up execution time on windows with solutions posted to [this issue](https://github.com/docker/for-win/issues/1936) +- If you run into `Permission Denied` errors see [this issue](https://github.com/docker/for-win/issues/3385#issuecomment-501931980) +- You can try to speed up execution time on windows with solutions posted to [this issue](https://github.com/docker/for-win/issues/1936) ### Coverage reports On the frontend, the lcov and HTML files are generated as normal, however on the backend, the folders are tested separately. The command `yarn coverage:backend` will concatenate the lcov files and also generate an HTML file. However, this provess requires `lcov` to be installed on a user's computer. On Apple, you can use Homebrew - `brew install lcov`. On a Windows machine, your path may vary, but two options include WSL and [this chocolatey package](https://community.chocolatey.org/packages/lcov). -Another important note for running tests on the backend - we specifically exclude files on the backend that follow the ```*CLI.js``` naming convention (for example, ```adminToolsCLI.js```) from test coverage. This is meant to exclude files intended to be run in the shell. Any functionality in theses files should be imported from a file that is tested. The ```src/tools folder``` is where these files have usually lived and there are lots of great examples of the desired pattern in that folder. +Another important note for running tests on the backend - we specifically exclude files on the backend that follow the `*CLI.js` naming convention (for example, `adminToolsCLI.js`) from test coverage. This is meant to exclude files intended to be run in the shell. Any functionality in theses files should be imported from a file that is tested. The `src/tools folder` is where these files have usually lived and there are lots of great examples of the desired pattern in that folder. ### Coverage reports: Uncovered lines on PR builds -The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. - * coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. - * uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. - * uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. +The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. - This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter ```fail-on-modified-lines``` to true, defaults to false. +- coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. +- uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. +- uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. + +This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter `fail-on-modified-lines` to true, defaults to false. ## Yarn Commands -| Docker Command | Description| Host Command | Local only Command | -|-|-|-|-| -| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | -| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | -| `yarn docker:stop` | Stops the backend and frontend | | | -| `yarn docker:dbs:start` | Start only the supporting services | | | -| `yarn docker:dbs:stop` | Stop only the supporting services | | | -| `yarn docker:test` | Runs tests for the frontend and backend | | | -| `yarn docker:lint` | Runs the linter for the frontend and backend | | | -| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | -| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | -| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | -| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | -| | Starts the backend web process | `yarn start:web` | `yarn server` | | -| | Starts the worker process | `yarn start:worker` | `yarn worker` | | -| | Start the frontend | | `yarn client` | -| | Run tests for only the backend | `yarn test`| | -| | Run tests for the backend with coverage and output results to xml files| `yarn test:ci`| | -| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all`| | -| | Run the linter only for the backend | `yarn lint` | | -| | Run the linter for the the backend with results output to xml files | `yarn lint:ci`| | -| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all`| | -| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | -| | Run cucumber tests | `yarn cucumber` | | -| | Collect backend coverage report | `yarn coverage:backend` || +| Docker Command | Description | Host Command | Local only Command | +| ----------------------------- | -------------------------------------------------------------------------------------------- | ----------------------- | ------------------ | --- | +| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | +| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | +| `yarn docker:stop` | Stops the backend and frontend | | | +| `yarn docker:dbs:start` | Start only the supporting services | | | +| `yarn docker:dbs:stop` | Stop only the supporting services | | | +| `yarn docker:test` | Runs tests for the frontend and backend | | | +| `yarn docker:lint` | Runs the linter for the frontend and backend | | | +| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | +| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | +| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | +| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | +| | Starts the backend web process | `yarn start:web` | `yarn server` | | +| | Starts the worker process | `yarn start:worker` | `yarn worker` | | +| | Start the frontend | | `yarn client` | +| | Run tests for only the backend | `yarn test` | | +| | Run tests for the backend with coverage and output results to xml files | `yarn test:ci` | | +| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all` | | +| | Run the linter only for the backend | `yarn lint` | | +| | Run the linter for the the backend with results output to xml files | `yarn lint:ci` | | +| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all` | | +| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | +| | Run cucumber tests | `yarn cucumber` | | +| | Collect backend coverage report | `yarn coverage:backend` | | ## Infrastructure ### Persistent vs Ephemeral Infrastructure The infrastructure used to run this application can be categorized into two distinct types: ephemeral and persistent. + - **Ephemeral infrastructure** is all the infrastructure that is recreated each time the application is deployed. Ephemeral infrastructure includes the "application" (as defined in Cloud.gov), the EC2 instances that application runs on, and the routes that application utilizes. This infrastructure is defined and deployed to Cloud.gov by the CircleCI configuration. - **Persistent infrastructure** is all the infrastructure that remains constant and unchanged despite application deployments. Persistent infrastructure includes the database used in each development environment. This infrastructure is defined and instantiated on Cloud.gov by the Terraform configuration files. For more about Terraform see [terraform/README.md](terraform/README.md). @@ -273,6 +274,7 @@ https://circleci.com/docs/github-integration/#create-a-github-deploy-key https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent Steps to create and apply deploy token: + 1. Open the Git Bash CMD window 2. Enter the following command with your github (admin) e-mail: ssh-keygen -t rsa -b 4096 -C "your_email@example.com" 3. When prompted to enter a file name leave blank and press ENTER @@ -300,18 +302,18 @@ variable files. Commits to specific branches in github repositories start automatic deployments as follows: -* **Dev** deployed by commit to [main branch][adhoc-main] of Ad Hoc fork of HHS/Head-Start-TTADP repo -* **Staging** deployed by commit to [main branch][hhs-main] of HHS/Head-Start-TTADP repo -* **Prod** deployed by commit to [production branch][hhs-prod] of HHS/Head-Start-TTADP repo +- **Dev** deployed by latest push to any remote branch of HHS/Head-Start-TTADP repo +- **Staging** deployed by commit to [main branch][hhs-main] of HHS/Head-Start-TTADP repo +- **Prod** deployed by commit to [production branch][hhs-prod] of HHS/Head-Start-TTADP repo The application is deployed to the following URLs: -| Environment | URL | -|:------------|:----| -| sandbox | https://tta-smarthub-sandbox.app.cloud.gov/ | -| dev | https://tta-smarthub-dev.app.cloud.gov/ | -| staging | https://tta-smarthub-staging.app.cloud.gov/ | -| prod | https://ttahub.ohs.acf.hhs.gov | +| Environment | URL | +| :---------- | :------------------------------------------ | +| sandbox | https://tta-smarthub-sandbox.app.cloud.gov/ | +| dev | https://tta-smarthub-dev.app.cloud.gov/ | +| staging | https://tta-smarthub-staging.app.cloud.gov/ | +| prod | https://ttahub.ohs.acf.hhs.gov | **Sandbox Environment** @@ -326,15 +328,17 @@ deleting the application. **Secret Management** CircleCI's project-based "environment variables" are used for secret management. These secrets include: + - Cloud.gov deployer account username and password. These keys are specific to each cloud.gov -organization/space, i.e. each deployment environment, and can be [regenerated by developers][cloudgov-deployer] -who have proper cloud.gov permissions at any time. + organization/space, i.e. each deployment environment, and can be [regenerated by developers][cloudgov-deployer] + who have proper cloud.gov permissions at any time. - HSES authentication middleware secrets passed to the application as `AUTH_CLIENT_ID` and `AUTH_CLIENT_SECRET`. - The application `SESSION_SECRET`. - The application `JWT_SECRET`. - NewRelic license key, passed to the application as `NEW_RELIC_LICENSE_KEY` Exception: + - The environment specific postgres database URI is automatically available in the relevant cloud.gov application environment (because they share a cloud.gov "space"). The URI is accessible to the application as POSTGRES_URL. Consequently, this secret does not need to be managed by developers. **Adding environment variables to an application** @@ -345,90 +349,92 @@ If your env variable is secret or the value is dependent on the deployment envir 1. If secret, add your variable to CircleCI - If the variable value you want to add needs to remain secret, you will need to add it as a project-based "environment variable" in CircleCI. Ad Hoc engineers can use [this link][circleci-envvar] to navigate to the Environment Variables page for our forked repository. Add your environment variables here. If you need different values for sandbox and dev make sure to make two variables, one for each environment. + If the variable value you want to add needs to remain secret, you will need to add it as a project-based "environment variable" in CircleCI. Ad Hoc engineers can use [this link][circleci-envvar] to navigate to the Environment Variables page for our forked repository. Add your environment variables here. If you need different values for sandbox and dev make sure to make two variables, one for each environment. - For example, if you needed to add an environment specific secret `SECRET_FRUIT` variable to your application, you could add `SANDBOX_SECRET_FRUIT` with value `strawberry` and `DEV_SECRET_FRUIT` with value `dewberry`. + For example, if you needed to add an environment specific secret `SECRET_FRUIT` variable to your application, you could add `SANDBOX_SECRET_FRUIT` with value `strawberry` and `DEV_SECRET_FRUIT` with value `dewberry`. 1. Add both secret and public variables to manifest.yml - In the application manifest, add your `SECRET_FRUIT` variable to the `env:` object. If you need another non-secret but environment specific variable, like `PUBLIC_VEGGIE`, in your application, add that here. + In the application manifest, add your `SECRET_FRUIT` variable to the `env:` object. If you need another non-secret but environment specific variable, like `PUBLIC_VEGGIE`, in your application, add that here. - **manifest.yml** + **manifest.yml** - ``` - --- - applications: - - name: tta-smarthub-((env)) - env: - SECRET_FRUIT: ((SECRET_FRUIT)) - PUBLIC_VEGGIE: ((public_veggie)) - ``` + ``` + --- + applications: + - name: tta-smarthub-((env)) + env: + SECRET_FRUIT: ((SECRET_FRUIT)) + PUBLIC_VEGGIE: ((public_veggie)) + ``` 1. If public, add the variable values to your deployment_config files - **deployment_config/sandbox_vars.yml** + **deployment_config/sandbox_vars.yml** - ``` - public_veggie: spinach - ``` + ``` + public_veggie: spinach + ``` - **deployment_config/dev_vars.yml** + **deployment_config/dev_vars.yml** - ``` - public_veggie: dill - ``` + ``` + public_veggie: dill + ``` - You're all done with public env variables! In sandbox, `process.env.PUBLIC_VEGGIE` will be `"spinach"`. In dev, `process.env.PUBLIC_VEGGIE` will be `"dill"`. 🎉 + You're all done with public env variables! In sandbox, `process.env.PUBLIC_VEGGIE` will be `"spinach"`. In dev, `process.env.PUBLIC_VEGGIE` will be `"dill"`. 🎉 1. If secret, pass your variables to the `cf_deploy` command in the circleci config. - Make two additions here: - - Add the variable under `parameters`. Give your variable a description and a type of `env_var_name`. - - Under `steps:`, pass your new parameter to `cf push` with the `--var` flag. You can think of `cf_push` as a function, which uses the `parameters` as inputs. Make sure to retain the `${}` syntax. This forces CircleCI to interpret your `secret_fruit` parameter as a project-based environment variable, and make the correct substitution. This will become clearer in the next step. - - **config/config.yml** - ``` - commands: - ... - cf_deploy: - ... - parameters: - ... - secret_fruit: - description: "Name of CircleCI project environment variable that - holds the secret fruit" - type: env_var_name - steps: - ... - - run: - name: Push application with deployment vars - command: | - cf push --vars-file << parameters.deploy_config_file >> \ - --var SECRET_FRUIT=${<< parameters.secret_fruit >>} - ``` + Make two additions here: + + - Add the variable under `parameters`. Give your variable a description and a type of `env_var_name`. + - Under `steps:`, pass your new parameter to `cf push` with the `--var` flag. You can think of `cf_push` as a function, which uses the `parameters` as inputs. Make sure to retain the `${}` syntax. This forces CircleCI to interpret your `secret_fruit` parameter as a project-based environment variable, and make the correct substitution. This will become clearer in the next step. + + **config/config.yml** + + ``` + commands: + ... + cf_deploy: + ... + parameters: + ... + secret_fruit: + description: "Name of CircleCI project environment variable that + holds the secret fruit" + type: env_var_name + steps: + ... + - run: + name: Push application with deployment vars + command: | + cf push --vars-file << parameters.deploy_config_file >> \ + --var SECRET_FRUIT=${<< parameters.secret_fruit >>} + ``` 1. If secret, in the `deploy` job, add the circle ci project environment variable name that you created in step 1 to the `cf_deploy` command as a parameter. - **config/config.yml** - ``` - jobs: - ... - deploy: - ... - when: # sandbox - ... - steps: - - cf_deploy: - secret_fruit: SANDBOX_SECRET_FRUIT - when: # dev - ... - steps: - - cf_deploy: - secret_fruit: DEV_SECRET_FRUIT - ``` - - You're all done! In sandbox, `process.env.SECRET_FRUIT` will be `"strawberry"`. In dev, `process.env.SECRET_FRUIT` will be `"dewberry"`. 🎉 - + **config/config.yml** + + ``` + jobs: + ... + deploy: + ... + when: # sandbox + ... + steps: + - cf_deploy: + secret_fruit: SANDBOX_SECRET_FRUIT + when: # dev + ... + steps: + - cf_deploy: + secret_fruit: DEV_SECRET_FRUIT + ``` + + You're all done! In sandbox, `process.env.SECRET_FRUIT` will be `"strawberry"`. In dev, `process.env.SECRET_FRUIT` will be `"dewberry"`. 🎉 ### Interacting with a deployed application or database @@ -440,15 +446,15 @@ Our project includes four deployed Postgres databases, one to interact with each 1. Install **Version 7** of the Cloud Foundry CLI tool - - On MacOS: `brew install cloudfoundry/tap/cf-cli@7` - - On other platforms: [Download and install cf][cf-install]. Be sure to get version 7.x + - On MacOS: `brew install cloudfoundry/tap/cf-cli@7` + - On other platforms: [Download and install cf][cf-install]. Be sure to get version 7.x 1. Login to cloud.gov account - ```bash - cf login -a api.fr.cloud.gov --sso - # follow temporary authorization code prompts - ``` + ```bash + cf login -a api.fr.cloud.gov --sso + # follow temporary authorization code prompts + ``` 1. Follow prompts to target the desired space @@ -458,81 +464,82 @@ Our project includes four deployed Postgres databases, one to interact with each 1. If you haven't used the the cloud foundry plugin [cf-service-connect][cf-service-connect] before, install it now - ```bash - # Example install for macOS - cf install-plugin https://github.com/cloud-gov/cf-service-connect/releases/download/1.1.0/cf-service-connect-darwin-386 - ``` + ```bash + # Example install for macOS + cf install-plugin https://github.com/cloud-gov/cf-service-connect/releases/download/1.1.0/cf-service-connect-darwin-386 + ``` 1. Connect to your desired database - ```bash - cf connect-to-service - # Example for sandbox - cf connect-to-service tta-smarthub-sandbox ttahub-sandbox - ``` + ```bash + cf connect-to-service + # Example for sandbox + cf connect-to-service tta-smarthub-sandbox ttahub-sandbox + ``` - On success, your terminal prompt will change to match the `db_name` from the database instance credentials. - This indicates you are in an open psql session, the command-line interface to PostgreSQL. + On success, your terminal prompt will change to match the `db_name` from the database instance credentials. + This indicates you are in an open psql session, the command-line interface to PostgreSQL. ##### Option B: Run script as task 1. Use [cf run-task][cf-run-task] command - ```bash - cf run-task --command "" - # Example 1: running data validation script against sandbox - cf run-task tta-smarthub-sandbox --command "yarn db:validation" - # Example 2: undo most recent database migration - cf run-task tta-smarthub-sandbox --command "yarn db:migrate:undo:prod:last" - ``` + ```bash + cf run-task --command "" + # Example 1: running data validation script against sandbox + cf run-task tta-smarthub-sandbox --command "yarn db:validation" + # Example 2: undo most recent database migration + cf run-task tta-smarthub-sandbox --command "yarn db:migrate:undo:prod:last" + ``` 1. Check log output, including those from task - ```bash - cf logs --recent - # Example 1: checking sandbox logs - cf logs tta-smarthub-sandbox --recent - # Example 2: checking sandbox logs, grep just for task logs - cf logs tta-smarthub-sandbox --recent | grep APP/TASK/ - ``` + ```bash + cf logs --recent + # Example 1: checking sandbox logs + cf logs tta-smarthub-sandbox --recent + # Example 2: checking sandbox logs, grep just for task logs + cf logs tta-smarthub-sandbox --recent | grep APP/TASK/ + ``` ##### Option C: Run script in an interactive shell 1. If on prod, enable shh in space first - ```bash - cf allow-space-ssh ttahub-prod - ``` + ```bash + cf allow-space-ssh ttahub-prod + ``` 1. Ssh into your desired application (to see application names run `cf apps`) - ```bash - cf ssh - # ssh example for sandbox application - cf ssh tta-smarthub-sandbox - ``` + ```bash + cf ssh + # ssh example for sandbox application + cf ssh tta-smarthub-sandbox + ``` 1. Open shell - ```bash - /tmp/lifecycle/shell - ``` + ```bash + /tmp/lifecycle/shell + ``` 1. Run your desired command - ```bash - # example - node ./build/server/src/tools/dataValidationCLI.js - ``` + ```bash + # example + node ./build/server/src/tools/dataValidationCLI.js + ``` 1. If on prod, disable ssh in space - ```bash - cf disallow-space-ssh ttahub-prod - ``` + ```bash + cf disallow-space-ssh ttahub-prod + ``` ### Taking a production backup via CircleCI -We can quickly take a production backup via the CircleCI web interface. To do so, go to the ```production``` branch there and trigger a pipeline with the variable ```manual-trigger``` set to true. You can then retrieve this backup with the script ```bin/latest_backup.sh```. + +We can quickly take a production backup via the CircleCI web interface. To do so, go to the `production` branch there and trigger a pipeline with the variable `manual-trigger` set to true. You can then retrieve this backup with the script `bin/latest_backup.sh`. ### Refreshing data in non-production environments @@ -543,6 +550,7 @@ version of the production database locally if using local database. The script c yarn processData:local ``` + The transformed database can then be restored in the non-production environments. For details on how to perform a backup and restore, there is information on the cloud.gov site: @@ -557,6 +565,7 @@ version of the production database locally if using local database. The script c yarn processData:local ``` + The transformed database can then be restored in the non-production environments. For details on how to perform a backup and restore, there is information on the cloud.gov site: @@ -569,12 +578,14 @@ if you need to put the application into maintenance mode, you can run the mainte This script require that you have [Cloud Foundry's CLI v7](https://github.com/cloudfoundry/cli/wiki/V7-CLI-Installation-Guide) installed to run. The script takes two flags + - \-m | \-\-maintenance\-mode controls whether the script takes the app into maintenance mode or out of it. - Options are "on" or "off - \-e | \-\-environment controls which environment you are targeting. - Options are "sandbox", "dev", "staging", and "prod" Ex. + ``` # Puts the dev environment into maintenance mode ./bin/maintenance -e dev -m on @@ -583,9 +594,11 @@ Ex. If you are not logged into the cf cli, it will ask you for an sso temporary password. You can get a temporary password at https://login.fr.cloud.gov/passcode. The application will stay in maintenance mode even through deploys of the application. You need to explicitly run `./bin/maintenance -e ${env} -m off` to turn off maintenance mode. ## Updating node + To update the version of node the project uses, the version number needs to be specified in a few places. Cloud.gov only supports certain versions of node; you can find supported versions [on the repo for their buildpack](https://github.com/cloudfoundry/nodejs-buildpack/releases). Once you have that version number, you need to update it in the following files + - .circleci/config.yml - .nvmrc - Dockerfile @@ -594,58 +607,58 @@ Once you have that version number, you need to update it in the following files You should also update it where it is specified this README file. -You would then need to rebuild the relevant browser images (docker will likely need to pull new ones) and run ```yarn docker:deps``` to rebuild your dependencies. -If you are using NVM, you can set intall a new node version with ```nvm install VERSION``` and set it to be the default version of node via ```nvm alias default VERSION```. +You would then need to rebuild the relevant browser images (docker will likely need to pull new ones) and run `yarn docker:deps` to rebuild your dependencies. +If you are using NVM, you can set intall a new node version with `nvm install VERSION` and set it to be the default version of node via `nvm alias default VERSION`. ## Removing, creating and binding a service from the command line -In the past, we've needed to destroy and recreate particular services (for example, redis). This can be done through the Cloud.gov UI, through the Terraform architecture, and through the cloud foundry command line interface. The following are instructions for using the cloud foundry CLI (```cf```) for this. -- Login and target the environment you wish to make changes to. (```cf login --sso```). -- You can use ```cf services``` to list your services -- Remember that you can use ```cf help COMMAND``` to get the documentation for a particular command +In the past, we've needed to destroy and recreate particular services (for example, redis). This can be done through the Cloud.gov UI, through the Terraform architecture, and through the cloud foundry command line interface. The following are instructions for using the cloud foundry CLI (`cf`) for this. + +- Login and target the environment you wish to make changes to. (`cf login --sso`). +- You can use `cf services` to list your services +- Remember that you can use `cf help COMMAND` to get the documentation for a particular command To delete and recreate a service (this should not be done lightly, as it is a destructive action) -1 Unbind a service: -```cf us APP_NAME SERVICE``` +1 Unbind a service: +`cf us APP_NAME SERVICE` ex: -```cf us tta-smarthub-staging ttahub-redis-staging``` +`cf us tta-smarthub-staging ttahub-redis-staging` -2 Delete a service: -```cf ds SERVICE``` +2 Delete a service: +`cf ds SERVICE` ex: -```cf ds ttahub-redis-staging``` +`cf ds ttahub-redis-staging` -3 Create a service: -```cf cs SERVICE_TYPE SERVICE_LABEL SERVICE``` +3 Create a service: +`cf cs SERVICE_TYPE SERVICE_LABEL SERVICE` ex: -```cf cs aws-elasticache-redis redis-dev ttahub-redis-staging``` +`cf cs aws-elasticache-redis redis-dev ttahub-redis-staging` -4 Bind a service: -```cf bs APP_NAME SERVICE``` +4 Bind a service: +`cf bs APP_NAME SERVICE` ex: -```cf bs ttahub-smarthub-staging ttahub-redis-staging``` +`cf bs ttahub-smarthub-staging ttahub-redis-staging` 5. Trigger a redeploy through the Circle CI UI (rather than restaging) 6. Finally, you may need to reconfigure the network policies to allow the app to connect to the virus scanning api. Check your network policies with: - ```cf network-policies``` -If you see nothing there, you'll need to add an appropriate policy. -```cf add-network-policy tta-smarthub-APP_NAME clamav-api-ttahub-APP_NAME --protocol tcp --port 9443``` -ex: -```cf add-network-policy tta-smarthub-dev clamav-api-ttahub-dev --protocol tcp --port 9443``` -You may need to connect across spaces (for example, our clamav-api-ttahub-dev app is shared by all of our ephemeral environments). If so, use the -s flag. -ex: -```cf add-network-policy tta-smarthub-staging -s ttahub-dev clamav-api-ttahub-dev --protocol tcp --port 9443``` - + `cf network-policies` + If you see nothing there, you'll need to add an appropriate policy. + `cf add-network-policy tta-smarthub-APP_NAME clamav-api-ttahub-APP_NAME --protocol tcp --port 9443` + ex: + `cf add-network-policy tta-smarthub-dev clamav-api-ttahub-dev --protocol tcp --port 9443` + You may need to connect across spaces (for example, our clamav-api-ttahub-dev app is shared by all of our ephemeral environments). If so, use the -s flag. + ex: + `cf add-network-policy tta-smarthub-staging -s ttahub-dev clamav-api-ttahub-dev --protocol tcp --port 9443` [Current tech stack](./tech-stack.md) -[adhoc-main]: https://github.com/adhocteam/Head-Start-TTADP/tree/main +[hhs-main]: https://github.com/HHS/Head-Start-TTADP/tree/main [TTAHUB-System-Operations]: https://github.com/HHS/Head-Start-TTADP/wiki/TTAHUB-System-Operations -[circleci-envvar]: https://app.circleci.com/settings/project/github/adhocteam/Head-Start-TTADP/environment-variables?return-to=https%3A%2F%2Fcircleci.com%2Fdashboard +[circleci-envvar]: https://app.circleci.com/settings/project/github/HHS/Head-Start-TTADP/environment-variables?return-to=https%3A%2F%2Fcircleci.com%2Fdashboard [cloudgov]: https://dashboard.fr.cloud.gov/home [cloudgov-deployer]: https://cloud.gov/docs/services/cloud-gov-service-account/ [cf-install]: https://docs.cloudfoundry.org/cf-cli/install-go-cli.html @@ -653,7 +666,7 @@ ex: [cf-service-connect]: https://github.com/cloud-gov/cf-service-connect [hhs-main]: https://github.com/HHS/Head-Start-TTADP/tree/main [hhs-prod]: https://github.com/HHS/Head-Start-TTADP/tree/production -[frontend]:http://localhost:3000 -[backend]:http://localhost:8080 -[API documentation]:http://localhost:5003 -[minio]:http://localhost:3000 +[frontend]: http://localhost:3000 +[backend]: http://localhost:8080 +[API documentation]: http://localhost:5003 +[minio]: http://localhost:3000 From b488e5d728e8ee21057decde1fcdfaf1ca7bdf84 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 14:19:14 -0800 Subject: [PATCH 02/17] cleanup prettier --- README.md | 64 +++++++++++++++++++++++++++---------------------------- 1 file changed, 31 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index ec048c79a4..93e661700b 100644 --- a/README.md +++ b/README.md @@ -96,17 +96,15 @@ You must also install and run minio locally to use the file upload functionality #### Precommit hooks -Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](https://github.com/HHS/Head-Start-TTADP/tree/main/terraform#README.md). +Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](terraform/README.md). If you are not using your own custom pre-commit hooks: -- **start from repo root directory** - -- **make the pre-commit file executable** - chmod 755 .githooks/pre-commit - -- **change your default hooks directory to `.githooks`.** - git config core.hooksPath .githooks +- start from repo root directory +- make the pre-commit file executable: + `chmod 755 .githooks/pre-commit` +- change your default hooks directory to `.githooks`: + `git config core.hooksPath .githooks` If you are already using git hooks, add the .githooks/pre-commit contents to your hooks directory or current pre-commit hook. Remember to make the file executable. @@ -224,31 +222,31 @@ This Uncovered lines on PR builds can be configured to fail builds by either per ## Yarn Commands -| Docker Command | Description | Host Command | Local only Command | -| ----------------------------- | -------------------------------------------------------------------------------------------- | ----------------------- | ------------------ | --- | -| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | -| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | -| `yarn docker:stop` | Stops the backend and frontend | | | -| `yarn docker:dbs:start` | Start only the supporting services | | | -| `yarn docker:dbs:stop` | Stop only the supporting services | | | -| `yarn docker:test` | Runs tests for the frontend and backend | | | -| `yarn docker:lint` | Runs the linter for the frontend and backend | | | -| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | -| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | -| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | -| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | -| | Starts the backend web process | `yarn start:web` | `yarn server` | | -| | Starts the worker process | `yarn start:worker` | `yarn worker` | | -| | Start the frontend | | `yarn client` | -| | Run tests for only the backend | `yarn test` | | -| | Run tests for the backend with coverage and output results to xml files | `yarn test:ci` | | -| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all` | | -| | Run the linter only for the backend | `yarn lint` | | -| | Run the linter for the the backend with results output to xml files | `yarn lint:ci` | | -| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all` | | -| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | -| | Run cucumber tests | `yarn cucumber` | | -| | Collect backend coverage report | `yarn coverage:backend` | | +| Docker Command | Description| Host Command | Local only Command | +| :-------- | :---------- | :---------- | :---------- | +| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | +| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | +| `yarn docker:stop` | Stops the backend and frontend | | | +| `yarn docker:dbs:start` | Start only the supporting services | | | +| `yarn docker:dbs:stop` | Stop only the supporting services | | | +| `yarn docker:test` | Runs tests for the frontend and backend | | | +| `yarn docker:lint` | Runs the linter for the frontend and backend | | | +| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | +| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | +| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | +| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | +| | Starts the backend web process | `yarn start:web` | `yarn server` | | +| | Starts the worker process | `yarn start:worker` | `yarn worker` | | +| | Start the frontend | | `yarn client` | +| | Run tests for only the backend | `yarn test`| | +| | Run tests for the backend with coverage and output results to xml files| `yarn test:ci`| | +| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all`| | +| | Run the linter only for the backend | `yarn lint` | | +| | Run the linter for the the backend with results output to xml files | `yarn lint:ci`| | +| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all`| | +| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | +| | Run cucumber tests | `yarn cucumber` | | +| | Collect backend coverage report | `yarn coverage:backend` || ## Infrastructure From cb163271af919032f856c38325bd12bfdb3c2802 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 14:44:12 -0800 Subject: [PATCH 03/17] fixup --- README.md | 54 +++++++++++++++++++++++++++--------------------------- 1 file changed, 27 insertions(+), 27 deletions(-) diff --git a/README.md b/README.md index 93e661700b..d56421310e 100644 --- a/README.md +++ b/README.md @@ -103,8 +103,8 @@ If you are not using your own custom pre-commit hooks: - start from repo root directory - make the pre-commit file executable: `chmod 755 .githooks/pre-commit` -- change your default hooks directory to `.githooks`: - `git config core.hooksPath .githooks` +- change your default hooks directory to [.githooks](.githooks)`: +`git config core.hooksPath .githooks` If you are already using git hooks, add the .githooks/pre-commit contents to your hooks directory or current pre-commit hook. Remember to make the file executable. @@ -222,31 +222,31 @@ This Uncovered lines on PR builds can be configured to fail builds by either per ## Yarn Commands -| Docker Command | Description| Host Command | Local only Command | -| :-------- | :---------- | :---------- | :---------- | -| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | -| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | -| `yarn docker:stop` | Stops the backend and frontend | | | -| `yarn docker:dbs:start` | Start only the supporting services | | | -| `yarn docker:dbs:stop` | Stop only the supporting services | | | -| `yarn docker:test` | Runs tests for the frontend and backend | | | -| `yarn docker:lint` | Runs the linter for the frontend and backend | | | -| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | -| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | -| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | -| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | -| | Starts the backend web process | `yarn start:web` | `yarn server` | | -| | Starts the worker process | `yarn start:worker` | `yarn worker` | | -| | Start the frontend | | `yarn client` | -| | Run tests for only the backend | `yarn test`| | -| | Run tests for the backend with coverage and output results to xml files| `yarn test:ci`| | -| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all`| | -| | Run the linter only for the backend | `yarn lint` | | -| | Run the linter for the the backend with results output to xml files | `yarn lint:ci`| | -| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all`| | -| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | -| | Run cucumber tests | `yarn cucumber` | | -| | Collect backend coverage report | `yarn coverage:backend` || +| Docker Command | Description | Host Command | Local only Command | +| :---------------------------- | :------------------------------------------------------------------------------------------- | :---------------------- | :----------------- | --- | +| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | +| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | +| `yarn docker:stop` | Stops the backend and frontend | | | +| `yarn docker:dbs:start` | Start only the supporting services | | | +| `yarn docker:dbs:stop` | Stop only the supporting services | | | +| `yarn docker:test` | Runs tests for the frontend and backend | | | +| `yarn docker:lint` | Runs the linter for the frontend and backend | | | +| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | +| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | +| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | +| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | +| | Starts the backend web process | `yarn start:web` | `yarn server` | | +| | Starts the worker process | `yarn start:worker` | `yarn worker` | | +| | Start the frontend | | `yarn client` | +| | Run tests for only the backend | `yarn test` | | +| | Run tests for the backend with coverage and output results to xml files | `yarn test:ci` | | +| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all` | | +| | Run the linter only for the backend | `yarn lint` | | +| | Run the linter for the the backend with results output to xml files | `yarn lint:ci` | | +| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all` | | +| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | +| | Run cucumber tests | `yarn cucumber` | | +| | Collect backend coverage report | `yarn coverage:backend` | | ## Infrastructure From ecccd6ad5c669fcc079a9fac0edf13640dba9b52 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:00:24 -0800 Subject: [PATCH 04/17] githooks --- README.md | 27 +++++++++++++++++---------- 1 file changed, 17 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 042efdcf85..5c09756b3b 100644 --- a/README.md +++ b/README.md @@ -23,9 +23,6 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr ### Set up -_Warning_ when using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of -those services are already running on your machine. - #### Docker 1. Make sure Docker is installed. To check run `docker ps`. @@ -33,14 +30,24 @@ those services are already running on your machine. 3. Copy `.env.example` to `.env`. 4. Change the `AUTH_CLIENT_ID` and `AUTH_CLIENT_SECRET` variables to to values found in the team Keybase account. If you don't have access to Keybase, please ask in the acf-head-start-eng slack channel for access. 5. Optionally, set `CURRENT_USER` to your current user's uid:gid. This will cause files created by docker compose to be owned by your user instead of root. -6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`. See the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. -7. Run `yarn docker:start` to start the application. The [frontend][frontend] will be available on `localhost:3000` and the [backend][backend] will run on `localhost:8080`, [API documentation][API documentation] will run on `localhost:5003`, and [minio][minio] will run on `localhost:9000`. +6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. +7. Run `yarn docker:start` to start the application. + +- The [frontend][frontend] will be available on `localhost:3000` \* +- The [backend][backend] will run on `localhost:8080` +- [API documentation][API documentation] will run on `localhost:5003` +- [minio][minio] (used for file storage) will run on `localhost:9000`. + 8. Run `yarn docker:stop` to stop the servers and remove the docker containers. -The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. +[!NOTE] Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. +The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. + +When using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of those services are already running on your machine. + #### Import Current Production Data Make sure you have access to all the necessary spaces on Cloud.gov @@ -103,10 +110,10 @@ If you are not using your own custom pre-commit hooks: - start from repo root directory - make the pre-commit file executable: `chmod 755 .githooks/pre-commit` -- change your default hooks directory to [.githooks](.githooks)`: -`git config core.hooksPath .githooks` +- change your default hooks directory to [.githooks](.githooks): + `git config core.hooksPath .githooks` -If you are already using git hooks, add the .githooks/pre-commit contents to your hooks directory or current pre-commit hook. Remember to make the file executable. +If you are already using git hooks, add the [.githooks/pre-commit](.githooks/pre-commit) contents to your hooks directory or current pre-commit hook. Remember to make the file executable. ### Building Tests @@ -536,8 +543,8 @@ Our project includes four deployed Postgres databases, one to interact with each ``` ##### Example: Manual import of Monitoring data -Importing Monitoring data without the automation uses Option C above across several step and is described further on in the [tools README](https://github.com/HHS/Head-Start-TTADP/tree/main/src/tools). +Importing Monitoring data without the automation uses Option C above across several step and is described further on in the [tools README](https://github.com/HHS/Head-Start-TTADP/tree/main/src/tools). ### Taking a production backup via CircleCI From 6800693d685765edfb318a511f678a69d1e78918 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:11:54 -0800 Subject: [PATCH 05/17] separate troubleshooting --- README.md | 63 ++++++++++++++++++++++++++++++------------------------- 1 file changed, 34 insertions(+), 29 deletions(-) diff --git a/README.md b/README.md index 5c09756b3b..2d243a274d 100644 --- a/README.md +++ b/README.md @@ -21,26 +21,29 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr ## Getting Started -### Set up +### Local Setup #### Docker -1. Make sure Docker is installed. To check run `docker ps`. -2. Make sure you have Node 18.20.6 installed. +[!NOTE] +If you run into issues during any of these steps, check the [troubleshooting](#notes-and-troubleshooting) section + +1. Install Docker. To check run `docker ps`. +2. Install Node, matching the version in [.nvmrc](.nvmrc). 3. Copy `.env.example` to `.env`. 4. Change the `AUTH_CLIENT_ID` and `AUTH_CLIENT_SECRET` variables to to values found in the team Keybase account. If you don't have access to Keybase, please ask in the acf-head-start-eng slack channel for access. 5. Optionally, set `CURRENT_USER` to your current user's uid:gid. This will cause files created by docker compose to be owned by your user instead of root. -6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. +6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. 7. Run `yarn docker:start` to start the application. -- The [frontend][frontend] will be available on `localhost:3000` \* -- The [backend][backend] will run on `localhost:8080` -- [API documentation][API documentation] will run on `localhost:5003` -- [minio][minio] (used for file storage) will run on `localhost:9000`. + - The [frontend][frontend] will run on `localhost:3000` + - The [backend][backend] will run on `localhost:8080` + - [API documentation][API documentation] will run on `localhost:5003` + - [minio][minio] (S3-compatible file storage) will run on `localhost:9000` 8. Run `yarn docker:stop` to stop the servers and remove the docker containers. -[!NOTE] +#### Notes and Troubleshooting Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. @@ -48,6 +51,28 @@ The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-r When using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of those services are already running on your machine. +##### Apple Silicon & Chromium + +If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. + +On a Mac with Apple Silicon, puppeteer install fails with the message: +`"The chromium binary is not available for arm64"` + +See [docker-compose.override.yml](docker-compose.override.yml) and uncomment the relevant lines to skip downloading chromium and use the host's binary instead. + +You will need to have chromium installed (you probably do not). The recommended installation method is to use brew: `brew install chromium --no-quarantine` + +To ~/.zshrc (or your particular shell config), you'll need to add: + +```sh +export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true +export PUPPETEER_EXECUTABLE_PATH=`which chromium` +``` + +On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: + +`brew install docker-compose` + #### Import Current Production Data Make sure you have access to all the necessary spaces on Cloud.gov @@ -75,26 +100,6 @@ create database ttasmarthub; On Windows TBD -#### Apple Silicon & Chromium - -On a Mac with Apple Silicon, puppeteer install fails with the message: -`"The chromium binary is not available for arm64"` - -See [docker-compose.override.yml](docker-compose.override.yml) and uncomment the relevant lines to skip downloading chromium and use the host's binary instead. - -You will need to have chromium installed (you probably do not). The recommended installation method is to use brew: `brew install chromium --no-quarantine` - -To ~/.zshrc (or your particular shell config), you'll need to add: - -```sh -export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true -export PUPPETEER_EXECUTABLE_PATH=`which chromium` -``` - -On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: - -`brew install docker-compose` - #### Local build You can also run build commands directly on your host (without docker). Make sure you install dependencies when changing execution method. You could see some odd errors if you install dependencies for docker and then run yarn commands directly on the host, especially if you are developing on windows. If you want to use the host yarn commands be sure to run `yarn deps:local` before any other yarn commands. Likewise if you want to use docker make sure you run `yarn docker:deps`. From 56c3c4962200271abf217d7db744318a2d9ecccb Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:12:51 -0800 Subject: [PATCH 06/17] fix note --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 2d243a274d..9ab8e49202 100644 --- a/README.md +++ b/README.md @@ -25,8 +25,8 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr #### Docker -[!NOTE] -If you run into issues during any of these steps, check the [troubleshooting](#notes-and-troubleshooting) section +> [!NOTE] +> If you run into issues during any of these steps, check the [troubleshooting](#notes-and-troubleshooting) section 1. Install Docker. To check run `docker ps`. 2. Install Node, matching the version in [.nvmrc](.nvmrc). From 0e6841a64ef32307b3844bca1ecb6975a46bf814 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:32:35 -0800 Subject: [PATCH 07/17] reorganize --- README.md | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/README.md b/README.md index 9ab8e49202..6c95084c09 100644 --- a/README.md +++ b/README.md @@ -19,14 +19,9 @@ Welcome to the home of the OHS TTAHUB. For the latest on our product mission, goals, initiatives, and KPIs, see the [Product Planning page](https://github.com/HHS/Head-Start-TTADP/wiki/Product-Planning). -## Getting Started +## Development Setup -### Local Setup - -#### Docker - -> [!NOTE] -> If you run into issues during any of these steps, check the [troubleshooting](#notes-and-troubleshooting) section +### Run With Docker 1. Install Docker. To check run `docker ps`. 2. Install Node, matching the version in [.nvmrc](.nvmrc). @@ -43,17 +38,25 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr 8. Run `yarn docker:stop` to stop the servers and remove the docker containers. -#### Notes and Troubleshooting +**Notes** + +The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. -The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. +#### Troubleshooting + +If you see errors that the version of nodejs is incorrect, you may have older versions of the containers built. +Delete those images and rerun ``yarn docker:reset`. When using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of those services are already running on your machine. -##### Apple Silicon & Chromium +On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: +`brew install docker-compose` + +**Apple Silicon & Chromium** -If this returns errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and it should rebuild them. If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. +If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. On a Mac with Apple Silicon, puppeteer install fails with the message: `"The chromium binary is not available for arm64"` @@ -69,9 +72,11 @@ export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true export PUPPETEER_EXECUTABLE_PATH=`which chromium` ``` -On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: +#### Run Natively -`brew install docker-compose` +You can also run build commands directly on your host (without docker). Make sure you install dependencies when changing execution method. You could see some odd errors if you install dependencies for docker and then run yarn commands directly on the host, especially if you are developing on windows. If you want to use the host yarn commands be sure to run `yarn deps:local` before any other yarn commands. Likewise if you want to use docker make sure you run `yarn docker:deps`. + +You must also install and run minio locally to use the file upload functionality. Please comment out `S3_ENDPOINT=http://minio:9000` and uncomment `S3_ENDPOINT=http://localhost:9000` in your .env file. #### Import Current Production Data @@ -100,12 +105,6 @@ create database ttasmarthub; On Windows TBD -#### Local build - -You can also run build commands directly on your host (without docker). Make sure you install dependencies when changing execution method. You could see some odd errors if you install dependencies for docker and then run yarn commands directly on the host, especially if you are developing on windows. If you want to use the host yarn commands be sure to run `yarn deps:local` before any other yarn commands. Likewise if you want to use docker make sure you run `yarn docker:deps`. - -You must also install and run minio locally to use the file upload functionality. Please comment out `S3_ENDPOINT=http://minio:9000` and uncomment `S3_ENDPOINT=http://localhost:9000` in your .env file. - #### Precommit hooks Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](terraform/README.md). From 339db4acab491f9c525d7e133f7b236b10093bde Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:50:35 -0800 Subject: [PATCH 08/17] reorg testing --- README.md | 85 +++++++++++++++++++++++++++---------------------------- 1 file changed, 42 insertions(+), 43 deletions(-) diff --git a/README.md b/README.md index 6c95084c09..06c29a1fa5 100644 --- a/README.md +++ b/README.md @@ -44,7 +44,7 @@ The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-r Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. -#### Troubleshooting +**Troubleshooting** If you see errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and rerun ``yarn docker:reset`. @@ -72,13 +72,13 @@ export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true export PUPPETEER_EXECUTABLE_PATH=`which chromium` ``` -#### Run Natively +### Run Natively You can also run build commands directly on your host (without docker). Make sure you install dependencies when changing execution method. You could see some odd errors if you install dependencies for docker and then run yarn commands directly on the host, especially if you are developing on windows. If you want to use the host yarn commands be sure to run `yarn deps:local` before any other yarn commands. Likewise if you want to use docker make sure you run `yarn docker:deps`. You must also install and run minio locally to use the file upload functionality. Please comment out `S3_ENDPOINT=http://minio:9000` and uncomment `S3_ENDPOINT=http://localhost:9000` in your .env file. -#### Import Current Production Data +### Import Current Production Data Make sure you have access to all the necessary spaces on Cloud.gov @@ -105,7 +105,7 @@ create database ttasmarthub; On Windows TBD -#### Precommit hooks +### Precommit hooks Our CI will fail if code is committed that doesn't pass our linter (eslint). This repository contains a pre-commit hook that runs eslint's built in "fix" command on all staged javascript files so that any autofixable errors will be fixed. The precommit hook, in .gihooks/pre-commit, also contains code to auto-format our terraform files, which you can read more about [here](terraform/README.md). @@ -119,6 +119,44 @@ If you are not using your own custom pre-commit hooks: If you are already using git hooks, add the [.githooks/pre-commit](.githooks/pre-commit) contents to your hooks directory or current pre-commit hook. Remember to make the file executable. +## Local Testing + +### Running Tests With Docker + +If switching branches for code review, run `yarn docker:reset` before running your tests. + +Run `yarn docker:test` to run all tests for the frontend and backend. + +To only run the frontend tests run `yarn docker:test frontend`. + +To only run the backend tests run `yarn docker:test backend`. + +Migrations and seeding of the test db occurs within the script run by the `docker:test` command. + +To run eslint run `yarn lint:all` or `yarn lint:fix:all` to have eslint attempt to fix linting problems. + +> [!NOTE] +> You may run into some issues running the docker commands on Windows: +> +> - If you run into `Permission Denied` errors see [this issue](https://github.com/docker/for-win/issues/3385#issuecomment-501931980) +> - You can try to speed up execution time on windows with solutions posted to [this issue](https://github.com/docker/for-win/issues/1936) + +### Coverage reports + +On the frontend, the lcov and HTML files are generated as normal, however on the backend, the folders are tested separately. The command `yarn coverage:backend` will concatenate the lcov files and also generate an HTML file. However, this provess requires `lcov` to be installed on a user's computer. On Apple, you can use Homebrew - `brew install lcov`. On a Windows machine, your path may vary, but two options include WSL and [this chocolatey package](https://community.chocolatey.org/packages/lcov). + +Another important note for running tests on the backend - we specifically exclude files on the backend that follow the `*CLI.js` naming convention (for example, `adminToolsCLI.js`) from test coverage. This is meant to exclude files intended to be run in the shell. Any functionality in theses files should be imported from a file that is tested. The `src/tools folder` is where these files have usually lived and there are lots of great examples of the desired pattern in that folder. + +#### Coverage reports: Uncovered lines on PR builds + +The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. + +- coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. +- uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. +- uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. + +This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter `fail-on-modified-lines` to true, defaults to false. + ### Building Tests #### Helpful notes on writing (backend) tests @@ -192,45 +230,6 @@ describe("Individual Test Isolation", () => { }); ``` -### Running Tests - -#### Docker - -If switching branches for code review, run `yarn docker:reset` before running your tests. - -Run `yarn docker:test` to run all tests for the frontend and backend. - -To only run the frontend tests run `yarn docker:test frontend`. - -To only run the backend tests run `yarn docker:test backend`. - -Migrations and seeding of the test db occurs within the script run by the `docker:test` command. - -To run eslint run `yarn lint:all` or `yarn lint:fix:all` to have eslint attempt to fix linting problems. - -### Docker on Windows - -You may run into some issues running the docker commands on Windows: - -- If you run into `Permission Denied` errors see [this issue](https://github.com/docker/for-win/issues/3385#issuecomment-501931980) -- You can try to speed up execution time on windows with solutions posted to [this issue](https://github.com/docker/for-win/issues/1936) - -### Coverage reports - -On the frontend, the lcov and HTML files are generated as normal, however on the backend, the folders are tested separately. The command `yarn coverage:backend` will concatenate the lcov files and also generate an HTML file. However, this provess requires `lcov` to be installed on a user's computer. On Apple, you can use Homebrew - `brew install lcov`. On a Windows machine, your path may vary, but two options include WSL and [this chocolatey package](https://community.chocolatey.org/packages/lcov). - -Another important note for running tests on the backend - we specifically exclude files on the backend that follow the `*CLI.js` naming convention (for example, `adminToolsCLI.js`) from test coverage. This is meant to exclude files intended to be run in the shell. Any functionality in theses files should be imported from a file that is tested. The `src/tools folder` is where these files have usually lived and there are lots of great examples of the desired pattern in that folder. - -### Coverage reports: Uncovered lines on PR builds - -The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. - -- coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. -- uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. -- uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. - -This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter `fail-on-modified-lines` to true, defaults to false. - ## Yarn Commands | Docker Command | Description | Host Command | Local only Command | From 9d35cf09e0c31f871f09dbb59e9b9535a8d3e58f Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:55:47 -0800 Subject: [PATCH 09/17] headings --- README.md | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 06c29a1fa5..010b371243 100644 --- a/README.md +++ b/README.md @@ -38,13 +38,12 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr 8. Run `yarn docker:stop` to stop the servers and remove the docker containers. -**Notes** - -The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. +> [!NOTE] +> The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. -Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the `docs/openapi` folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. +> Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the [docs/openapi](docs/openapi) folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. -**Troubleshooting** +### Troubleshooting If you see errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and rerun ``yarn docker:reset`. @@ -54,7 +53,7 @@ When using Docker to run either the full app or the backend services, PostgreSQL On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: `brew install docker-compose` -**Apple Silicon & Chromium** +#### Apple Silicon & Chromium If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. @@ -121,7 +120,7 @@ If you are already using git hooks, add the [.githooks/pre-commit](.githooks/pre ## Local Testing -### Running Tests With Docker +### Running Tests If switching branches for code review, run `yarn docker:reset` before running your tests. From 9624c557d136e9af522513ad6c181d709ad47944 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 15:58:41 -0800 Subject: [PATCH 10/17] down a level --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 010b371243..79b92891fe 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr ## Development Setup -### Run With Docker +#### Run With Docker 1. Install Docker. To check run `docker ps`. 2. Install Node, matching the version in [.nvmrc](.nvmrc). @@ -43,7 +43,7 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr > Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the [docs/openapi](docs/openapi) folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. -### Troubleshooting +#### Troubleshooting If you see errors that the version of nodejs is incorrect, you may have older versions of the containers built. Delete those images and rerun ``yarn docker:reset`. @@ -53,7 +53,7 @@ When using Docker to run either the full app or the backend services, PostgreSQL On a Mac with Brew installed Docker, yarn commands may fail due to the absence of `docker-compose` (vs `docker compose`). To resolve: `brew install docker-compose` -#### Apple Silicon & Chromium +**Apple Silicon & Chromium** If you are using a newer Mac with the Apple Silicon chipset, puppeteer install fails with the message: `"The chromium binary is not available for arm64"`, see the section immediately following this one, entitled "Apple Silicon & Chromium" for instructions on how to proceed. @@ -71,7 +71,7 @@ export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true export PUPPETEER_EXECUTABLE_PATH=`which chromium` ``` -### Run Natively +#### Run Natively You can also run build commands directly on your host (without docker). Make sure you install dependencies when changing execution method. You could see some odd errors if you install dependencies for docker and then run yarn commands directly on the host, especially if you are developing on windows. If you want to use the host yarn commands be sure to run `yarn deps:local` before any other yarn commands. Likewise if you want to use docker make sure you run `yarn docker:deps`. From b477222f4e63ec3ac25c0b4713a629a6512d4335 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 16:01:21 -0800 Subject: [PATCH 11/17] formatting --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 79b92891fe..7323daae43 100644 --- a/README.md +++ b/README.md @@ -19,10 +19,12 @@ Welcome to the home of the OHS TTAHUB. For the latest on our product mission, goals, initiatives, and KPIs, see the [Product Planning page](https://github.com/HHS/Head-Start-TTADP/wiki/Product-Planning). -## Development Setup +## Getting Started #### Run With Docker +You can run the application stack with Docker by following these steps: + 1. Install Docker. To check run `docker ps`. 2. Install Node, matching the version in [.nvmrc](.nvmrc). 3. Copy `.env.example` to `.env`. From 1078246aa8e33a813682f1ed9e40249e83724e96 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 16:26:40 -0800 Subject: [PATCH 12/17] tables in ui --- README.md | 67 +++++++++++++++++++++++++++---------------------------- 1 file changed, 33 insertions(+), 34 deletions(-) diff --git a/README.md b/README.md index 7323daae43..3dfb5d2d39 100644 --- a/README.md +++ b/README.md @@ -21,9 +21,9 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr ## Getting Started -#### Run With Docker +### Run With Docker -You can run the application stack with Docker by following these steps: +If you run into issues, check the [troubleshooting](#troubleshooting) section. 1. Install Docker. To check run `docker ps`. 2. Install Node, matching the version in [.nvmrc](.nvmrc). @@ -32,23 +32,21 @@ You can run the application stack with Docker by following these steps: 5. Optionally, set `CURRENT_USER` to your current user's uid:gid. This will cause files created by docker compose to be owned by your user instead of root. 6. Run `yarn docker:reset`. This builds the frontend and backend, installs dependencies, then runs database migrations and seeders. 7. Run `yarn docker:start` to start the application. - - The [frontend][frontend] will run on `localhost:3000` - The [backend][backend] will run on `localhost:8080` - [API documentation][API documentation] will run on `localhost:5003` - [minio][minio] (S3-compatible file storage) will run on `localhost:9000` - 8. Run `yarn docker:stop` to stop the servers and remove the docker containers. -> [!NOTE] -> The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. +**Notes:** -> Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the [docs/openapi](docs/openapi) folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. +- The frontend [proxies requests](https://create-react-app.dev/docs/proxying-api-requests-in-development/) to paths it doesn't recognize to the backend. +- Api documentation uses [Redoc](https://github.com/Redocly/redoc) to serve documentation files. These files can be found in the [docs/openapi](docs/openapi) folder. Api documentation should be split into separate files when appropriate to prevent huge hard to grasp yaml files. #### Troubleshooting If you see errors that the version of nodejs is incorrect, you may have older versions of the containers built. -Delete those images and rerun ``yarn docker:reset`. +Delete those images and rerun `yarn docker:reset`. When using Docker to run either the full app or the backend services, PostgreSQL (5432) and Redis (6379) are both configured to bind to their well-known ports. This will fail if any other instances of those services are already running on your machine. @@ -95,7 +93,7 @@ On a Mac 8. Edit .env and change CURRENT_USER_ID= from 1 to the ID of a production user 9. Restart docker -bounce.sql +_bounce.sql_ ```sh select pg_terminate_backend(pid) from pg_stat_activity where datname='ttasmarthub'; @@ -233,31 +231,32 @@ describe("Individual Test Isolation", () => { ## Yarn Commands -| Docker Command | Description | Host Command | Local only Command | -| :---------------------------- | :------------------------------------------------------------------------------------------- | :---------------------- | :----------------- | --- | -| `yarn docker:deps` | Install dependencies for the frontend and backend | `yarn deps` | `yarn deps:local` | -| `yarn docker:start` | Starts the backend and frontend | | `yarn start:local` | -| `yarn docker:stop` | Stops the backend and frontend | | | -| `yarn docker:dbs:start` | Start only the supporting services | | | -| `yarn docker:dbs:stop` | Stop only the supporting services | | | -| `yarn docker:test` | Runs tests for the frontend and backend | | | -| `yarn docker:lint` | Runs the linter for the frontend and backend | | | -| `yarn docker:db:migrate` | Run migrations in docker containers | `yarn db:migrate` | | -| `yarn docker:db:migrate:undo` | Undo migrations in docker containers | `yarn db:migrate:undo` | | -| `yarn docker:db:seed` | Run all seeders located in `src/seeders` | `yarn db:seed` | | -| `yarn docker:db:seed:undo` | Undo all seeders located in `src/seeders` | `yarn db:seed:undo` | | -| | Starts the backend web process | `yarn start:web` | `yarn server` | | -| | Starts the worker process | `yarn start:worker` | `yarn worker` | | -| | Start the frontend | | `yarn client` | -| | Run tests for only the backend | `yarn test` | | -| | Run tests for the backend with coverage and output results to xml files | `yarn test:ci` | | -| | Run `yarn test:ci` for both the frontend and backend | `yarn test:all` | | -| | Run the linter only for the backend | `yarn lint` | | -| | Run the linter for the the backend with results output to xml files | `yarn lint:ci` | | -| | Run `yarn lint:ci` for both the frontend and backend | `yarn lint:all` | | -| | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | `yarn docs:serve` | | -| | Run cucumber tests | `yarn cucumber` | | -| | Collect backend coverage report | `yarn coverage:backend` | | +// prettier-ignore +| Description | Docker Command | Host Command | Local only Command | +|-|-|-|-| +| Install dependencies for the frontend and backend | `yarn docker:deps` |`yarn deps` | `yarn deps:local` | +| Starts the backend and frontend | `yarn docker:start` | | `yarn start:local` | +| Stops the backend and frontend | `yarn docker:stop` | | | +| Start only the supporting services | | `yarn docker:dbs:start` | | +| Stop only the supporting services | `yarn docker:dbs:stop` || | +| Runs tests for the frontend and backend | `yarn docker:test` | | | +| Runs the linter for the frontend and backend | `yarn docker:lint` | | | +| Run migrations in docker containers | `yarn docker:db:migrate` | `yarn db:migrate` | | +| Undo migrations in docker containers | `yarn docker:db:migrate:undo` | `yarn db:migrate:undo` | | +| Run all seeders located in `src/seeders` | `yarn docker:db:seed` | `yarn db:seed` | | +| Undo all seeders located in `src/seeders` | `yarn docker:db:seed:undo` | `yarn db:seed:undo` | | +| Starts the backend web process | | `yarn start:web` | `yarn server` | | +| Starts the worker process | | `yarn start:worker` | `yarn worker` | | +| Start the frontend | | | `yarn client` | +| Run tests for only the backend | | `yarn test`| | +| Run tests for the backend with coverage and output results to xml files| | `yarn test:ci`| | +| Run `yarn test:ci` for both the frontend and backend | | `yarn test:all`| | +| Run the linter only for the backend | | `yarn lint` | | +| Run the linter for the the backend with results output to xml files | | `yarn lint:ci`| | +| Run `yarn lint:ci` for both the frontend and backend | | `yarn lint:all`| | +| Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | | `yarn docs:serve` | | +| Run cucumber tests | | `yarn cucumber` | | +| Collect backend coverage report | | `yarn coverage:backend` || ## Infrastructure From 694c2af8cf17f15daf6c3b29c26ace32769152f2 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 16:27:29 -0800 Subject: [PATCH 13/17] wording --- README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 3dfb5d2d39..000668c8f9 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ For the latest on our product mission, goals, initiatives, and KPIs, see the [Pr ## Getting Started -### Run With Docker +### Running With Docker If you run into issues, check the [troubleshooting](#troubleshooting) section. @@ -71,7 +71,7 @@ export PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true export PUPPETEER_EXECUTABLE_PATH=`which chromium` ``` -#### Run Natively +#### Running Natively You can also run build commands directly on your host (without docker). Make sure you install dependencies when changing execution method. You could see some odd errors if you install dependencies for docker and then run yarn commands directly on the host, especially if you are developing on windows. If you want to use the host yarn commands be sure to run `yarn deps:local` before any other yarn commands. Likewise if you want to use docker make sure you run `yarn docker:deps`. From fb613243c15138af8d3fd08d39fd3dda993f3d55 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 16:44:33 -0800 Subject: [PATCH 14/17] try this --- README.md | 71 ++++++++++++++++++++++++++++++------------------------- 1 file changed, 39 insertions(+), 32 deletions(-) diff --git a/README.md b/README.md index 000668c8f9..ca6862d488 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,11 @@ Welcome to the home of the OHS TTAHUB. For the latest on our product mission, goals, initiatives, and KPIs, see the [Product Planning page](https://github.com/HHS/Head-Start-TTADP/wiki/Product-Planning). +[Local Development](#getting-started) +[Testing](#getting-started) +[Infrastructure](#infrastructure) +[Reference](#reference) + ## Getting Started ### Running With Docker @@ -118,7 +123,7 @@ If you are not using your own custom pre-commit hooks: If you are already using git hooks, add the [.githooks/pre-commit](.githooks/pre-commit) contents to your hooks directory or current pre-commit hook. Remember to make the file executable. -## Local Testing +## Testing ### Running Tests @@ -229,35 +234,6 @@ describe("Individual Test Isolation", () => { }); ``` -## Yarn Commands - -// prettier-ignore -| Description | Docker Command | Host Command | Local only Command | -|-|-|-|-| -| Install dependencies for the frontend and backend | `yarn docker:deps` |`yarn deps` | `yarn deps:local` | -| Starts the backend and frontend | `yarn docker:start` | | `yarn start:local` | -| Stops the backend and frontend | `yarn docker:stop` | | | -| Start only the supporting services | | `yarn docker:dbs:start` | | -| Stop only the supporting services | `yarn docker:dbs:stop` || | -| Runs tests for the frontend and backend | `yarn docker:test` | | | -| Runs the linter for the frontend and backend | `yarn docker:lint` | | | -| Run migrations in docker containers | `yarn docker:db:migrate` | `yarn db:migrate` | | -| Undo migrations in docker containers | `yarn docker:db:migrate:undo` | `yarn db:migrate:undo` | | -| Run all seeders located in `src/seeders` | `yarn docker:db:seed` | `yarn db:seed` | | -| Undo all seeders located in `src/seeders` | `yarn docker:db:seed:undo` | `yarn db:seed:undo` | | -| Starts the backend web process | | `yarn start:web` | `yarn server` | | -| Starts the worker process | | `yarn start:worker` | `yarn worker` | | -| Start the frontend | | | `yarn client` | -| Run tests for only the backend | | `yarn test`| | -| Run tests for the backend with coverage and output results to xml files| | `yarn test:ci`| | -| Run `yarn test:ci` for both the frontend and backend | | `yarn test:all`| | -| Run the linter only for the backend | | `yarn lint` | | -| Run the linter for the the backend with results output to xml files | | `yarn lint:ci`| | -| Run `yarn lint:ci` for both the frontend and backend | | `yarn lint:all`| | -| Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | | `yarn docs:serve` | | -| Run cucumber tests | | `yarn cucumber` | | -| Collect backend coverage report | | `yarn coverage:backend` || - ## Infrastructure ### Persistent vs Ephemeral Infrastructure @@ -605,7 +581,38 @@ Ex. If you are not logged into the cf cli, it will ask you for an sso temporary password. You can get a temporary password at https://login.fr.cloud.gov/passcode. The application will stay in maintenance mode even through deploys of the application. You need to explicitly run `./bin/maintenance -e ${env} -m off` to turn off maintenance mode. -## Updating node +## Reference + +### Yarn Commands + +// prettier-ignore +| Description | Docker Command | Host Command | Local only Command | +|-|-|-|-| +| Install dependencies for the frontend and backend | `yarn docker:deps` |`yarn deps` | `yarn deps:local` | +| Starts the backend and frontend | `yarn docker:start` | | `yarn start:local` | +| Stops the backend and frontend | `yarn docker:stop` | | | +| Start only the supporting services | | `yarn docker:dbs:start` | | +| Stop only the supporting services | `yarn docker:dbs:stop` || | +| Runs tests for the frontend and backend | `yarn docker:test` | | | +| Runs the linter for the frontend and backend | `yarn docker:lint` | | | +| Run migrations in docker containers | `yarn docker:db:migrate` | `yarn db:migrate` | | +| Undo migrations in docker containers | `yarn docker:db:migrate:undo` | `yarn db:migrate:undo` | | +| Run all seeders located in `src/seeders` | `yarn docker:db:seed` | `yarn db:seed` | | +| Undo all seeders located in `src/seeders` | `yarn docker:db:seed:undo` | `yarn db:seed:undo` | | +| Starts the backend web process | | `yarn start:web` | `yarn server` | | +| Starts the worker process | | `yarn start:worker` | `yarn worker` | | +| Start the frontend | | | `yarn client` | +| Run tests for only the backend | | `yarn test`| | +| Run tests for the backend with coverage and output results to xml files| | `yarn test:ci`| | +| Run `yarn test:ci` for both the frontend and backend | | `yarn test:all`| | +| Run the linter only for the backend | | `yarn lint` | | +| Run the linter for the the backend with results output to xml files | | `yarn lint:ci`| | +| Run `yarn lint:ci` for both the frontend and backend | | `yarn lint:all`| | +| Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | | `yarn docs:serve` | | +| Run cucumber tests | | `yarn cucumber` | | +| Collect backend coverage report | | `yarn coverage:backend` || + +### Updating node To update the version of node the project uses, the version number needs to be specified in a few places. Cloud.gov only supports certain versions of node; you can find supported versions [on the repo for their buildpack](https://github.com/cloudfoundry/nodejs-buildpack/releases). @@ -622,7 +629,7 @@ You should also update it where it is specified this README file. You would then need to rebuild the relevant browser images (docker will likely need to pull new ones) and run `yarn docker:deps` to rebuild your dependencies. If you are using NVM, you can set intall a new node version with `nvm install VERSION` and set it to be the default version of node via `nvm alias default VERSION`. -## Removing, creating and binding a service from the command line +### Removing, creating and binding a service from the command line In the past, we've needed to destroy and recreate particular services (for example, redis). This can be done through the Cloud.gov UI, through the Terraform architecture, and through the cloud foundry command line interface. The following are instructions for using the cloud foundry CLI (`cf`) for this. From 48c639ffb0c183a5dcf0e1c98416d62f536d1a1d Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 16:45:09 -0800 Subject: [PATCH 15/17] toc --- README.md | 50 ++++++++++++++++++++++++++------------------------ 1 file changed, 26 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index ca6862d488..c65bf3ca54 100644 --- a/README.md +++ b/README.md @@ -19,15 +19,18 @@ Welcome to the home of the OHS TTAHUB. For the latest on our product mission, goals, initiatives, and KPIs, see the [Product Planning page](https://github.com/HHS/Head-Start-TTADP/wiki/Product-Planning). -[Local Development](#getting-started) -[Testing](#getting-started) -[Infrastructure](#infrastructure) -[Reference](#reference) - ## Getting Started +- [Tech Stack](./tech-stack.md) +- [Dev Setup](#getting-started) +- [Testing](#testing) +- [Infrastructure](#infrastructure) +- [Reference](#reference) + ### Running With Docker +For a full list of available yarn commands, see [here](#yarn-commands) + If you run into issues, check the [troubleshooting](#troubleshooting) section. 1. Install Docker. To check run `docker ps`. @@ -145,22 +148,6 @@ To run eslint run `yarn lint:all` or `yarn lint:fix:all` to have eslint attempt > - If you run into `Permission Denied` errors see [this issue](https://github.com/docker/for-win/issues/3385#issuecomment-501931980) > - You can try to speed up execution time on windows with solutions posted to [this issue](https://github.com/docker/for-win/issues/1936) -### Coverage reports - -On the frontend, the lcov and HTML files are generated as normal, however on the backend, the folders are tested separately. The command `yarn coverage:backend` will concatenate the lcov files and also generate an HTML file. However, this provess requires `lcov` to be installed on a user's computer. On Apple, you can use Homebrew - `brew install lcov`. On a Windows machine, your path may vary, but two options include WSL and [this chocolatey package](https://community.chocolatey.org/packages/lcov). - -Another important note for running tests on the backend - we specifically exclude files on the backend that follow the `*CLI.js` naming convention (for example, `adminToolsCLI.js`) from test coverage. This is meant to exclude files intended to be run in the shell. Any functionality in theses files should be imported from a file that is tested. The `src/tools folder` is where these files have usually lived and there are lots of great examples of the desired pattern in that folder. - -#### Coverage reports: Uncovered lines on PR builds - -The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. - -- coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. -- uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. -- uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. - -This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter `fail-on-modified-lines` to true, defaults to false. - ### Building Tests #### Helpful notes on writing (backend) tests @@ -234,6 +221,22 @@ describe("Individual Test Isolation", () => { }); ``` +### Coverage reports + +On the frontend, the lcov and HTML files are generated as normal, however on the backend, the folders are tested separately. The command `yarn coverage:backend` will concatenate the lcov files and also generate an HTML file. However, this provess requires `lcov` to be installed on a user's computer. On Apple, you can use Homebrew - `brew install lcov`. On a Windows machine, your path may vary, but two options include WSL and [this chocolatey package](https://community.chocolatey.org/packages/lcov). + +Another important note for running tests on the backend - we specifically exclude files on the backend that follow the `*CLI.js` naming convention (for example, `adminToolsCLI.js`) from test coverage. This is meant to exclude files intended to be run in the shell. Any functionality in theses files should be imported from a file that is tested. The `src/tools folder` is where these files have usually lived and there are lots of great examples of the desired pattern in that folder. + +#### Coverage reports: Uncovered lines on PR builds + +The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. + +- coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. +- uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. +- uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. + +This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter `fail-on-modified-lines` to true, defaults to false. + ## Infrastructure ### Persistent vs Ephemeral Infrastructure @@ -585,7 +588,7 @@ If you are not logged into the cf cli, it will ask you for an sso temporary pass ### Yarn Commands -// prettier-ignore + | Description | Docker Command | Host Command | Local only Command | |-|-|-|-| | Install dependencies for the frontend and backend | `yarn docker:deps` |`yarn deps` | `yarn deps:local` | @@ -611,6 +614,7 @@ If you are not logged into the cf cli, it will ask you for an sso temporary pass | Host the open api 3 spec using [redoc](https://github.com/Redocly/redoc) at `localhost:5003` | | `yarn docs:serve` | | | Run cucumber tests | | `yarn cucumber` | | | Collect backend coverage report | | `yarn coverage:backend` || + ### Updating node @@ -673,8 +677,6 @@ ex: -[Current tech stack](./tech-stack.md) - [hhs-main]: https://github.com/HHS/Head-Start-TTADP/tree/main [TTAHUB-System-Operations]: https://github.com/HHS/Head-Start-TTADP/wiki/TTAHUB-System-Operations [circleci-envvar]: https://app.circleci.com/settings/project/github/HHS/Head-Start-TTADP/environment-variables?return-to=https%3A%2F%2Fcircleci.com%2Fdashboard From 60c4c8d09e55a4eab8c91d7a3b2adf09ab6413d2 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Thu, 13 Feb 2025 16:51:18 -0800 Subject: [PATCH 16/17] looking better --- README.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index c65bf3ca54..84cbe8aeb1 100644 --- a/README.md +++ b/README.md @@ -231,9 +231,9 @@ Another important note for running tests on the backend - we specifically exclud The uncovered lines on PR is generated by finding the intersection between the jest generated coverage file with the git change list for the PR. The additional set of artifacts is generated to aid in providing test coverage for each PR. -- coverage/coverage-final.json - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. -- uncovered-lines/uncovered-lines.html - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. -- uncovered-lines/uncovered-lines.json - A json structure identifing all the lines from this PR that are uncovered by jest tests. +- [coverage/coverage-final.json](coverage/coverage-final.json) - Only on test_backend, all the distinct jest run outputs are consolidated into a unified coverage-final.json file. +- [uncovered-lines/uncovered-lines.html](uncovered-lines/uncovered-lines.html) - A human readable structure identifing all the lines from this PR that are uncovered by jest tests. +- [uncovered-lines/uncovered-lines.json](uncovered-lines/uncovered-lines.json) - A json structure identifing all the lines from this PR that are uncovered by jest tests. This Uncovered lines on PR builds can be configured to fail builds by either perminently changing or overiding the pipeline perameter `fail-on-modified-lines` to true, defaults to false. @@ -538,8 +538,7 @@ In order to keep the non-production environments as close to production as possi version of the production database locally if using local database. The script can be run using the following: ``` - yarn processData:local - +yarn processData:local ``` The transformed database can then be restored in the non-production environments. @@ -553,8 +552,7 @@ In order to keep the non-production environments as close to production as possi version of the production database locally if using local database. The script can be run using the following: ``` - yarn processData:local - +yarn processData:local ``` The transformed database can then be restored in the non-production environments. From 8663a051afc25bcc5c585d396f3238f11d418b28 Mon Sep 17 00:00:00 2001 From: Tom Meagher Date: Tue, 18 Feb 2025 11:25:38 -0800 Subject: [PATCH 17/17] remove more adhoc refs, fixup product wiki --- README.md | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index 84cbe8aeb1..57680ce690 100644 --- a/README.md +++ b/README.md @@ -17,7 +17,7 @@ Welcome to the home of the OHS TTAHUB. ## What We're Building and Why -For the latest on our product mission, goals, initiatives, and KPIs, see the [Product Planning page](https://github.com/HHS/Head-Start-TTADP/wiki/Product-Planning). +For the latest on our product mission, goals, initiatives, and KPIs, see the [Product Planning page](https://github.com/HHS/Head-Start-TTADP/wiki/Product-Roadmapping). ## Getting Started @@ -250,7 +250,7 @@ The infrastructure used to run this application can be categorized into two dist #### Continuous Integration (CI) -Linting, unit tests, test coverage analysis, and an accessibility scan are all run automatically on each push to the Ad Hoc fork of HHS/Head-Start-TTADP repo and the HHS/Head-Start-TTADP repo. In the Ad Hoc repository, merges to the main branch are blocked if the CI tests do not pass. The continuous integration pipeline is configured via CircleCi. The bulk of CI configurations can be found in this repo's [.circleci/config.yml](.circleci/config.yml) file. For more information on the security audit and scan tools used in the continuous integration pipeline see [ADR 0009](docs/adr/0009-security-scans.md). +Linting, unit tests, test coverage analysis, and an accessibility scan are all run automatically on each push to the HHS/Head-Start-TTADP repo. Merges to the main branch are blocked if the CI tests do not pass. The continuous integration pipeline is configured via CircleCi. The bulk of CI configurations can be found in this repo's [.circleci/config.yml](.circleci/config.yml) file. For more information on the security audit and scan tools used in the continuous integration pipeline see [ADR 0009](docs/adr/0009-security-scans.md). #### Creating and Applying a Deploy Key @@ -305,12 +305,7 @@ The application is deployed to the following URLs: **Sandbox Environment** An additional deployment environment called "sandbox" is available to developers for testing and -development on feature branches prior to making a commit to Ad Hoc's main branch and deploying to "dev". -The current [.circleci/config.yml](.circleci/config.yml) assumes deployments to this space are being made from -the Ad Hoc repository. Deployments are pushed to the ohstta-sandbox cloud.gov space. To conserve -resources, running application instances pushed to this space should be deleted as soon as they are no longer needed. -Running application instances can be deleted by logging into [cloud.gov][cloudgov], and then selecting and -deleting the application. +development on feature branches prior to making a commit to the main branch. Deployments are pushed to the ohstta-sandbox cloud.gov space. To conserve resources, running application instances pushed to this space should be deleted as soon as they are no longer needed. Running application instances can be deleted by logging into [cloud.gov][cloudgov], and then selecting and deleting the application. **Secret Management**