Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-1842] [TT-1608] Parrot Server #1595

Merged
merged 33 commits into from
Jan 24, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
23df5c7
In progress
kalverra Jan 10, 2025
4429324
WIP
kalverra Jan 10, 2025
1b3b249
Merge
kalverra Jan 13, 2025
0da4359
First draft and rename
kalverra Jan 14, 2025
e2f838f
Rename
kalverra Jan 14, 2025
da8385e
Rename workflows
kalverra Jan 14, 2025
c6218da
Fixes tests adds examples
kalverra Jan 15, 2025
60c0500
Docs and race test
kalverra Jan 15, 2025
ae77491
Beginning of recorder
kalverra Jan 15, 2025
79453fa
Better logging
kalverra Jan 16, 2025
b523706
Adds recorder
kalverra Jan 16, 2025
42dda5f
Better middleware
kalverra Jan 23, 2025
9598eda
Fixes header writing
kalverra Jan 23, 2025
93ad4a1
Fixes recorders
kalverra Jan 23, 2025
c25306f
Examples
kalverra Jan 23, 2025
ed2e7a8
Better save files
kalverra Jan 23, 2025
47685ce
Merge branch 'main' of github.com:smartcontractkit/chainlink-testing-…
kalverra Jan 23, 2025
5c2f4bd
More tests on recorders
kalverra Jan 23, 2025
d4ac2fa
Fix goreleaser
kalverra Jan 23, 2025
26c8b03
Goreleaser monorepo
kalverra Jan 23, 2025
af6f589
Merge branch 'main' of github.com:smartcontractkit/chainlink-testing-…
kalverra Jan 23, 2025
00d262d
Update upload artifact
kalverra Jan 23, 2025
f75a8d9
Update docs
kalverra Jan 23, 2025
6a3a569
Fix example name
kalverra Jan 23, 2025
e205865
Add linting
kalverra Jan 23, 2025
8a73802
Remove typo
kalverra Jan 23, 2025
d44b362
Adds example for external recorder
kalverra Jan 24, 2025
9f0ef71
Update README
kalverra Jan 24, 2025
a6bda1f
Fix goreleaser
kalverra Jan 24, 2025
207d9fe
More docs
kalverra Jan 24, 2025
378e65a
Code cleanup and more tests
kalverra Jan 24, 2025
c22fa42
Better debugging
kalverra Jan 24, 2025
336d1fc
Fix artifact name
kalverra Jan 24, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/docker-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ jobs:
go test -timeout 20m -json -parallel 2 -cover -covermode=atomic -coverprofile=unit-test-coverage.out $(go list ./... | grep /docker/test_env) -run '${{ matrix.test.tests }}' 2>&1 | tee /tmp/gotest.log | ../gotestloghelper -ci
- name: Publish Artifacts
if: failure()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@v4
with:
name: test-logs
path: ./lib/logs
8 changes: 4 additions & 4 deletions .github/workflows/framework-golden-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
config: smoke.toml
count: 1
timeout: 10m
- name: TestSmoke
- name: TestSmokeLimitedResources
config: smoke_limited_resources.toml
count: 1
timeout: 10m
Expand Down Expand Up @@ -81,9 +81,9 @@ jobs:
- 'framework/**'
- '.github/workflows/framework-golden-tests.yml'
- name: Set up Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: 1.22.8
go-version: 1.23
- name: Cache Go modules
uses: actions/cache@v3
with:
Expand All @@ -104,7 +104,7 @@ jobs:
go test -timeout ${{ matrix.test.timeout }} -v -count ${{ matrix.test.count }} -run ${{ matrix.test.name }}
- name: Upload Logs
if: always()
uses: actions/upload-artifact@v3
uses: actions/upload-artifact@v4
with:
name: container-logs-${{ matrix.test.name }}
path: framework/examples/myproject/logs
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/framework.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,9 @@ jobs:
src:
- 'framework/**'
- name: Set up Go
uses: actions/setup-go@v4
uses: actions/setup-go@v5
with:
go-version: 1.22.8
go-version: 1.23
- name: Cache Go modules
uses: actions/cache@v4
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/generate-go-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@
shell: bash
env:
OPENAI_API_KEY: ${{ secrets.OPENAI_DOC_GEN_API_KEY }}
run: |

Check failure on line 114 in .github/workflows/generate-go-docs.yaml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/generate-go-docs.yaml#L114

shellcheck reported issue in this script: SC2002:style:4:5: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead [shellcheck]
Raw output
.github/workflows/generate-go-docs.yaml:114:9: shellcheck reported issue in this script: SC2002:style:4:5: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead [shellcheck]
# Add go binary to PATH
PATH=$PATH:$(go env GOPATH)/bin
export PATH
Expand All @@ -123,7 +123,7 @@
rm filtered_folders.json

- name: Upload costs as artifact
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@v4
with:
name: generation-costs
path: ./costs
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/k8s-e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ jobs:
QA_AWS_ROLE_TO_ASSUME: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
run_setup: false
- name: Upload test log
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@v4
if: failure()
with:
name: test-log
Expand Down Expand Up @@ -146,7 +146,7 @@ jobs:
QA_KUBECONFIG: ${{ secrets.QA_KUBECONFIG }}
run_setup: false
- name: Upload test log
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@v4
if: failure()
with:
name: remote-runner-test-log
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,8 @@ jobs:
path: ./tools/asciitable/
- name: workflowresultparser
path: ./tools/workflowresultparser/
- name: parrot
path: ./parrot/
steps:
- name: Check out Code
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1
Expand All @@ -98,7 +100,7 @@ jobs:
run: test -f ${{ matrix.project.path }}golangci-lint-report.xml || true
- name: Store lint report artifact
if: always()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@v4
with:
name: golangci-lint-report-${{ matrix.project.name }}
path: ${{ matrix.project.path }}golangci-lint-report.xml
Expand Down
44 changes: 44 additions & 0 deletions .github/workflows/parrot-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Parrotserver Release

on:
push:
tags:
- parrot/v*

jobs:
release:
name: Build and Release
runs-on: ubuntu-latest
environment: integration
steps:
- name: Checkout repo
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1
with:
fetch-depth: 0
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@e3dd6a429d7300a6a4c196c26e071d42e0343502 # v4.0.2
with:
aws-region: ${{ secrets.QA_AWS_REGION }}
role-to-assume: ${{ secrets.QA_AWS_ROLE_TO_ASSUME }}
role-duration-seconds: 600
- name: Login to Amazon ECR
uses: aws-actions/amazon-ecr-login@062b18b96a7aff071d4dc91bc00c4c1a7945b076 # v2.0.1
with:
mask-password: 'true'
env:
AWS_REGION: ${{ secrets.QA_AWS_REGION }}
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: stable
- name: Goreleaser Release
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser-pro
version: "~> v2"
args: release --clean -f ./parrot/.goreleaser.yml
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
GORELEASER_KEY: ${{ secrets.GORELEASER_KEY }}
IMAGE_PREFIX: ${{ secrets.QA_AWS_ACCOUNT_NUMBER }}.dkr.ecr.${{ secrets.QA_AWS_REGION }}.amazonaws.com/parrot
IMAGE_TAG: ${{ github.ref_name}}
4 changes: 2 additions & 2 deletions .github/workflows/rc-breaking-changes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ jobs:
with:
fetch-depth: 0
fetch-tags: true
- name: Set up Go 1.23.3
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23.3'
go-version: 1.23
- name: Install gorelease tool
run: |
go install golang.org/x/exp/cmd/gorelease@latest
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-go-module.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@
echo "VERSION=$VERSION" >> "$GITHUB_ENV"
- name: Find Last Tag for Package and Generate Release Notes
id: generate_release_notes
run: |

Check failure on line 43 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L43

shellcheck reported issue in this script: SC2046:warning:2:79: Quote this to prevent word splitting [shellcheck]
Raw output
.github/workflows/release-go-module.yml:43:9: shellcheck reported issue in this script: SC2046:warning:2:79: Quote this to prevent word splitting [shellcheck]

Check failure on line 43 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L43

shellcheck reported issue in this script: SC2129:style:23:1: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
Raw output
.github/workflows/release-go-module.yml:43:9: shellcheck reported issue in this script: SC2129:style:23:1: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
# Find the latest tag for the same package that is not the current tag
LAST_TAG=$(git describe --abbrev=0 --always --match "$PACKAGE_NAME/v*" --tags $(git rev-list --tags --skip=1 --max-count=1))
echo "Last tag: ${LAST_TAG}"
Expand Down Expand Up @@ -69,14 +69,14 @@
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: '1.23.3'
go-version: 1.23
- name: Install gorelease tool
run: |
go install golang.org/x/exp/cmd/gorelease@latest
- name: Run gorelease to check for breaking changes
working-directory: ${{ env.PACKAGE_NAME }}
id: check_breaking_changes
run: |

Check failure on line 79 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L79

shellcheck reported issue in this script: SC2086:info:4:73: Double quote to prevent globbing and word splitting [shellcheck]
Raw output
.github/workflows/release-go-module.yml:79:9: shellcheck reported issue in this script: SC2086:info:4:73: Double quote to prevent globbing and word splitting [shellcheck]

Check failure on line 79 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L79

shellcheck reported issue in this script: SC2129:style:7:1: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
Raw output
.github/workflows/release-go-module.yml:79:9: shellcheck reported issue in this script: SC2129:style:7:1: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
set +e # Disable exit on error to capture output even if the command fails
echo "Last tag version: ${{ env.LAST_TAG_VERSION }}"
echo "Current tag version: ${VERSION}"
Expand All @@ -90,7 +90,7 @@
if: always()
working-directory: ${{ env.PACKAGE_NAME }}
id: read_additional_notes
run: |

Check failure on line 93 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L93

shellcheck reported issue in this script: SC2129:style:7:3: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
Raw output
.github/workflows/release-go-module.yml:93:9: shellcheck reported issue in this script: SC2129:style:7:3: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]

Check failure on line 93 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L93

shellcheck reported issue in this script: SC2086:info:13:14: Double quote to prevent globbing and word splitting [shellcheck]
Raw output
.github/workflows/release-go-module.yml:93:9: shellcheck reported issue in this script: SC2086:info:13:14: Double quote to prevent globbing and word splitting [shellcheck]

Check failure on line 93 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L93

shellcheck reported issue in this script: SC2129:style:14:3: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
Raw output
.github/workflows/release-go-module.yml:93:9: shellcheck reported issue in this script: SC2129:style:14:3: Consider using { cmd1; cmd2; } >> file instead of individual redirects [shellcheck]
# Check if the .changeset directory exists and the file for the current version is present
if [ -f ".changeset/${{ env.VERSION }}.md" ]; then
# Read the content of the file
Expand Down Expand Up @@ -127,7 +127,7 @@
echo "CMD_ENTRYPOINT_EXISTS=false" >> "$GITHUB_ENV"
fi
- name: Set binary name based on PACKAGE_NAME
run: |

Check failure on line 130 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L130

shellcheck reported issue in this script: SC2086:info:2:29: Double quote to prevent globbing and word splitting [shellcheck]
Raw output
.github/workflows/release-go-module.yml:130:9: shellcheck reported issue in this script: SC2086:info:2:29: Double quote to prevent globbing and word splitting [shellcheck]

Check failure on line 130 in .github/workflows/release-go-module.yml

View workflow job for this annotation

GitHub Actions / actionlint

[actionlint] .github/workflows/release-go-module.yml#L130

shellcheck reported issue in this script: SC2086:info:4:49: Double quote to prevent globbing and word splitting [shellcheck]
Raw output
.github/workflows/release-go-module.yml:130:9: shellcheck reported issue in this script: SC2086:info:4:49: Double quote to prevent globbing and word splitting [shellcheck]
if [ "${{ env.PACKAGE_NAME }}" == "framework" ]; then
echo "BINARY_NAME=ctf" >> $GITHUB_ENV
else
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ jobs:
path: ./tools/flakeguard/
- name: workflowresultparser
path: ./tools/workflowresultparser/
- name: parrot
path: ./parrot/
runs-on: ubuntu-latest
name: ${{ matrix.project.name }} unit tests
steps:
Expand All @@ -52,7 +54,7 @@ jobs:
make test_unit"
- name: Publish Artifacts
if: failure()
uses: actions/upload-artifact@b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882 # v4.4.3
uses: actions/upload-artifact@v4
with:
name: test-logs
path: /tmp/gotest.log
7 changes: 6 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -77,4 +77,9 @@ __debug*
.tool-versions

import_keys_test.go
tag.py
tag.py

parrot/*.json
parrot/*.log
# Executable
parrot/parrot
13 changes: 13 additions & 0 deletions parrot/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
Dockerfile
*.md
*.log
.gitignore
.golangci-lint.yml
.goreleaser.yml
.pre-commit-config.yaml
*_test.go
LICENSE
.vscode/
dist/
.github/
save.json
59 changes: 59 additions & 0 deletions parrot/.goreleaser.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# yaml-language-server: $schema=https://goreleaser.com/static/schema-pro.json
version: 2
project_name: parrot

monorepo:
tag_prefix: parrot/
dir: parrot

env:
- IMG_PRE={{ if index .Env "IMAGE_PREFIX" }}{{ .Env.IMAGE_PREFIX }}{{ else }}local{{ end }}
- TAG={{ if index .Env "IMAGE_TAG" }}{{ .Env.IMAGE_TAG }}{{ else }}latest{{ end }}

# Build settings for binaries
builds:
- id: parrot
main: ./cmd/main.go
goos:
- linux
- darwin
goarch:
- amd64
- arm64
ldflags:
- '-s -w'

archives:
- formats: ['binary']

dockers:
- id: linux-amd64-parrot
goos: linux
goarch: amd64
image_templates:
- '{{ .Env.IMG_PRE }}/parrot:{{ .Tag }}'
- '{{ .Env.IMG_PRE }}/parrot:latest'
build_flag_templates:
- --platform=linux/amd64
- --pull
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.title={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}
- id: linux-arm64-parrot
goos: linux
goarch: arm64
image_templates:
- '{{ .Env.IMG_PRE }}/parrot:{{ .Tag }}-arm64'
- '{{ .Env.IMG_PRE }}/parrot:latest-arm64'
build_flag_templates:
- --platform=linux/arm64
- --pull
- --label=org.opencontainers.image.created={{.Date}}
- --label=org.opencontainers.image.title={{.ProjectName}}
- --label=org.opencontainers.image.revision={{.FullCommit}}
- --label=org.opencontainers.image.version={{.Version}}

before:
hooks:
- cd parrot && go mod tidy
3 changes: 3 additions & 0 deletions parrot/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
FROM scratch
COPY parrotserver /parrotserver
ENTRYPOINT [ "parrotserver" ]
37 changes: 37 additions & 0 deletions parrot/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Default test log level (can be overridden)
PARROT_TEST_LOG_LEVEL ?= ""

# Pass TEST_LOG_LEVEL as a flag to go test
TEST_ARGS ?= -testLogLevel=$(PARROT_TEST_LOG_LEVEL)

.PHONY: lint
lint:
golangci-lint --color=always run ./... --fix -v

.PHONY: test
test:
go install github.com/gotesttools/gotestfmt/v2/cmd/gotestfmt@latest
set -euo pipefail
go test $(TEST_ARGS) -json -cover -coverprofile cover.out -v ./... 2>&1 | tee /tmp/gotest.log | gotestfmt

.PHONY: test_race
test_race:
go install github.com/gotesttools/gotestfmt/v2/cmd/gotestfmt@latest
set -euo pipefail
go test $(TEST_ARGS) -json -cover -count=1 -race -coverprofile cover.out -v ./... 2>&1 | tee /tmp/gotest.log | gotestfmt

.PHONY: test_unit
test_unit:
go test $(TEST_ARGS) -coverprofile cover.out ./...

.PHONY: bench
bench:
go test $(TEST_ARGS) -bench=. -run=^$$ ./...

.PHONY: build
build:
go build -o ./parrot ./cmd

.PHONY: goreleaser
goreleaser:
cd .. && goreleaser build --snapshot --clean -f ./parrot/.goreleaser.yaml
35 changes: 35 additions & 0 deletions parrot/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Parrot Server

A simple, high-performing mockserver that can dynamically build new routes with customized responses, parroting back whatever you tell it to.

## Features

* Simplistic and fast design
* Run within your Go code, through a small binary, or in a minimal Docker container
* Easily record all incoming requests to the server to programmatically react to

## Use

See our runnable examples in [examples_test.go](./examples_test.go) to see how to use Parrot programmatically.

## Run

```sh
go run ./cmd
go run ./cmd -h # See all config options
```

## Test

```sh
make test
make test PARROT_TEST_LOG_LEVEL=trace # Set log level for tests
make test_race # Test with -race flag enabled
make bench # Benchmark
```

## Build

```sh
make goreleaser # Uses goreleaser to build binaries and docker containers
```
Loading
Loading