Skip to content

Commit 4c249a4

Browse files
general-kroll-4-lifegeneral-kroll-4-life
committed
best-effort-integration-testing
Summary: - Better documentation for aws assume role auth model. - Added replica best effort `robot` integration test suite for foreign tests. - `aws` tests only to begin with.
1 parent 0ff5315 commit 4c249a4

8 files changed

Lines changed: 184 additions & 2 deletions

File tree

‎.github/workflows/build.yml‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -745,6 +745,42 @@ jobs:
745745
run: |
746746
cat ./test/robot/integration-traffic-lights/tmp/* || true
747747
748+
- name: Run foreign traffic light robot integration tests
749+
if: ((startsWith(github.ref_name, 'build-traffic-lights') && github.ref_type == 'tag') || (github.repository == 'stackql/stackql' && github.event_name == 'push' && github.ref == 'refs/heads/main')) && matrix.registry == 'test/registry'
750+
env:
751+
PYTHONPATH: '${{ env.PYTHONPATH }}:${{ github.workspace }}/test/python'
752+
AZURE_CLIENT_ID: ${{ secrets.AZURE_CLIENT_ID }}
753+
AZURE_CLIENT_SECRET: ${{ secrets.AZURE_CLIENT_SECRET }}
754+
AZURE_INTEGRATION_TESTING_SUB_ID: ${{ secrets.AZURE_INTEGRATION_TESTING_SUB_ID }}
755+
AZURE_TENANT_ID: ${{ secrets.AZURE_TENANT_ID }}
756+
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_FOREIGN_ACCESS_KEY_ID }}
757+
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_FOREIGN_SECRET_ACCESS_KEY }}
758+
STACKQL_AUDIT_ROLE_ARN: ${{ secrets.STACKQL_AUDIT_ROLE_ARN }}
759+
GODEBUG: netdns=go
760+
run: |
761+
echo "## Stray flask apps to be killed before robot tests ##"
762+
pgrep -f flask | xargs kill -9 || true
763+
echo "## End ##"
764+
if python cicd/python/build.py --robot-test-foreign-traffic-lights-integration; then
765+
echo "✅ Foreign traffic light robot integration tests **all** passed"
766+
else
767+
rv="$?"
768+
echo "🟡 **some** foreign traffic light robot integration tests failed code = $rv"
769+
fi
770+
{
771+
echo "FOREIGN_TRAFFIC_LIGHTS_COMPLETED=true"
772+
} >> "$GITHUB_ENV"
773+
774+
- name: Output from foreign traffic lights integration tests
775+
if: env.FOREIGN_TRAFFIC_LIGHTS_COMPLETED == 'true'
776+
run: |
777+
cat ./test/robot/reports-foreign-integration-traffic-lights/output.xml || true
778+
779+
- name: Output from foreign traffic lights tmp dir
780+
if: env.FOREIGN_TRAFFIC_LIGHTS_COMPLETED == 'true'
781+
run: |
782+
cat ./test/robot/foreign-integration-traffic-lights/tmp/* || true
783+
748784
- name: Generate rewritten registry for simulations
749785
if: ${{ matrix.registry != 'test/registry' }}
750786
run: |

‎cicd/python/build.py‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,16 @@ def run_robot_integration_traffic_lights_tests_stackql(*args, **kwargs) -> int:
8282
shell=True
8383
)
8484

85+
def run_robot_foreign_integration_traffic_lights_tests_stackql(*args, **kwargs) -> int:
86+
variables = ' '.join([f'--variable {key}:{sanitise_val(value)} ' for key, value in kwargs.get("variables", {}).items()])
87+
return subprocess.call(
88+
'robot '
89+
f'{variables} '
90+
'-d test/robot/reports-foreign-integration-traffic-lights '
91+
'test/robot/foreign-integration-traffic-lights',
92+
shell=True
93+
)
94+
8595
def main():
8696
parser = argparse.ArgumentParser()
8797
parser.add_argument('--verbose', action='store_true')
@@ -91,6 +101,7 @@ def main():
91101
parser.add_argument('--robot-test', action='store_true')
92102
parser.add_argument('--robot-test-integration', action='store_true')
93103
parser.add_argument('--robot-test-traffic-lights-integration', action='store_true')
104+
parser.add_argument('--robot-test-foreign-traffic-lights-integration', action='store_true')
94105
parser.add_argument('--config', type=json.loads, default={})
95106
args = parser.parse_args()
96107
ret_code = 0
@@ -118,6 +129,10 @@ def main():
118129
ret_code = run_robot_integration_traffic_lights_tests_stackql(**args.config)
119130
if ret_code != 0:
120131
exit(ret_code)
132+
if args.robot_test_foreign_traffic_lights_integration:
133+
ret_code = run_robot_foreign_integration_traffic_lights_tests_stackql(**args.config)
134+
if ret_code != 0:
135+
exit(ret_code)
121136
exit(ret_code)
122137

123138

‎docs/auth.md‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,8 +44,11 @@ The `--auth` `json` string can be configured for assuming a foriegn role. Tis,
4444
{"aws": {"type": "aws_assume_role", "keyIDenvvar": "AWS_ACCESS_KEY_ID", "credentialsenvvar": "AWS_SECRET_ACCESS_KEY", "aws_role_arn": "arn:aws:iam::123456789012:role/MyRole"}}
4545
```
4646

47-
Eg:
47+
Eg, presuming the source scripts constains the cited env vars:
4848

4949
```bash
50-
stackql --auth '{"aws":{"type":"aws_assume_role","keyIDenvvar":"AWS_ACCESS_KEY_ID","credentialsenvvar":"AWS_SECRET_ACCESS_KEY", "aws_role_arn": "arn:aws:iam::123456789012:role/MyRole"}}' shell
50+
51+
source cicd/vol/vendor-secrets/ryuk_to_stackql_user.sh
52+
53+
stackql --auth '{"aws":{"type":"aws_assume_role","keyIDenvvar":"AWS_ACCESS_KEY_ID","credentialsenvvar":"AWS_SECRET_ACCESS_KEY", "aws_role_arn": "'${STACKQL_AUDIT_ROLE_ARN}'"}}' shell
5154
```

‎test/robot/foreign-integration-traffic-lights/__init__.robot‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
*** Settings ***
2+
Resource ${CURDIR}/stackql.resource
3+
4+

‎test/robot/foreign-integration-traffic-lights/stackql.resource‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
*** Variables ***
2+
${LOCAL_LIB_HOME} ${CURDIR}${/}..${/}..${/}python
3+
${REPOSITORY_ROOT} ${CURDIR}${/}..${/}..${/}..
4+
${EXECUTION_PLATFORM} native # to be overridden from command line, eg "docker"
5+
${SQL_BACKEND} sqlite_embedded # to be overridden from command line, eg "postgres_tcp"
6+
${IS_WSL} false # to be overridden from command line, with string "true"
7+
${USE_STACKQL_PREINSTALLED} false # to be overridden from command line, with string "true"
8+
${SUNDRY_CONFIG} {} # to be overridden from command line, with string value
9+
${STACKQL_INTERFACE_LIBRARY} stackql_test_tooling.StackQLInterfaces
10+
${CLOUD_INTEGRATION_LIBRARY} stackql_test_tooling.CloudIntegration
11+
12+
*** Settings ***
13+
Library Process
14+
Library OperatingSystem
15+
Variables ${LOCAL_LIB_HOME}/stackql_test_tooling/stackql_context.py ${REPOSITORY_ROOT} ${EXECUTION_PLATFORM} ${SQL_BACKEND} ${USE_STACKQL_PREINSTALLED}
16+
... ${SUNDRY_CONFIG}
17+
Library Process
18+
Library OperatingSystem
19+
Library String
20+
Library ${STACKQL_INTERFACE_LIBRARY} ${EXECUTION_PLATFORM} ${SQL_BACKEND}
21+
Library ${CLOUD_INTEGRATION_LIBRARY}

‎test/robot/foreign-integration-traffic-lights/stackql_foreign_traffic_light_integration_from_cmd_line.robot‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
2+
3+
4+
*** Settings ***
5+
Resource ${CURDIR}/stackql.resource
6+
7+
*** Test Cases ***
8+
Nop From Lib
9+
${result} = Nop Cloud Integration Keyword
10+
Should Be Equal ${result} PASS
11+
12+
13+
AWS S3 Buckets Location Constraint
14+
Sleep 2s
15+
${awsRoleArn} = Get Environment Variable STACKQL_AUDIT_ROLE_ARN
16+
Should Not Be Empty ${awsRoleArn}
17+
${awsAuthCfg} = Catenate
18+
... { "aws": { "type":"aws_assume_role", "keyIDenvvar": "AWS_ACCESS_KEY_ID", "credentialsenvvar": "AWS_SECRET_ACCESS_KEY", "aws_role_arn": "${awsRoleArn}" } }
19+
${locactionConstraintQuery} = Catenate
20+
... select LocationConstraint from aws.s3.bucket_locations where region = 'ap-southeast-1' and Bucket = 'stackql-trial-bucket-01';
21+
${result} = Run Process
22+
... ${STACKQL_EXE}
23+
... \-\-auth
24+
... ${awsAuthCfg}
25+
... \-\-registry
26+
... { "url": "file://${REPOSITORY_ROOT}/test/registry", "localDocRoot": "${REPOSITORY_ROOT}/test/registry", "verifyConfig": { "nopVerify": true } }
27+
... exec
28+
... ${locactionConstraintQuery}
29+
... cwd=${REPOSITORY_ROOT}
30+
... stdout=${CURDIR}/tmp/AWS-S3-Buckets-Location-Constraint.tmp
31+
... stderr=${CURDIR}/tmp/AWS-S3-Buckets-Location-Constraint-stderr.tmp
32+
Should Be Equal As Integers ${result.rc} 0
33+
Should Contain ${result.stdout} ap\-southeast\-1
34+
35+
AWS S3 Buckets List
36+
Sleep 2s
37+
${awsRoleArn} = Get Environment Variable STACKQL_AUDIT_ROLE_ARN
38+
Should Not Be Empty ${awsRoleArn}
39+
${awsAuthCfg} = Catenate
40+
... { "aws": { "type":"aws_assume_role", "keyIDenvvar": "AWS_ACCESS_KEY_ID", "credentialsenvvar": "AWS_SECRET_ACCESS_KEY", "aws_role_arn": "${awsRoleArn}" } }
41+
${bucketsListQuery} = Catenate
42+
... select * from aws.s3.buckets where region = 'us-east-1' order by BucketArn desc;
43+
${result} = Run Process
44+
... ${STACKQL_EXE}
45+
... \-\-auth
46+
... ${awsAuthCfg}
47+
... \-\-registry
48+
... { "url": "file://${REPOSITORY_ROOT}/test/registry", "localDocRoot": "${REPOSITORY_ROOT}/test/registry", "verifyConfig": { "nopVerify": true } }
49+
... exec
50+
... ${bucketsListQuery}
51+
... cwd=${REPOSITORY_ROOT}
52+
... stdout=${CURDIR}/tmp/AWS-S3-Buckets-List.tmp
53+
... stderr=${CURDIR}/tmp/AWS-S3-Buckets-List-stderr.tmp
54+
Should Be Equal As Integers ${result.rc} 0
55+
Should Contain ${result.stdout} stackql\-trial\-bucket\-02
56+
57+
AWS S3 Bucket Objects List
58+
Sleep 2s
59+
${awsRoleArn} = Get Environment Variable STACKQL_AUDIT_ROLE_ARN
60+
Should Not Be Empty ${awsRoleArn}
61+
${awsAuthCfg} = Catenate
62+
... { "aws": { "type":"aws_assume_role", "keyIDenvvar": "AWS_ACCESS_KEY_ID", "credentialsenvvar": "AWS_SECRET_ACCESS_KEY", "aws_role_arn": "${awsRoleArn}" } }
63+
${bucketObjectsListQuery} = Catenate
64+
... select * from aws.s3.objects where Bucket = 'stackql-trial-bucket-02' and region = 'ap-southeast-2';
65+
${result} = Run Process
66+
... ${STACKQL_EXE}
67+
... \-\-auth
68+
... ${awsAuthCfg}
69+
... \-\-registry
70+
... { "url": "file://${REPOSITORY_ROOT}/test/registry", "localDocRoot": "${REPOSITORY_ROOT}/test/registry", "verifyConfig": { "nopVerify": true } }
71+
... exec
72+
... ${bucketObjectsListQuery}
73+
... cwd=${REPOSITORY_ROOT}
74+
... stdout=${CURDIR}/tmp/AWS-S3-Bucket-Objects-List.tmp
75+
... stderr=${CURDIR}/tmp/AWS-S3-Bucket-Objects-List-stderr.tmp
76+
Should Be Equal As Integers ${result.rc} 0
77+
Should Contain ${result.stdout} docs/advanced
78+
79+
AWS S3 Bucket ABAC Works
80+
Sleep 2s
81+
${awsRoleArn} = Get Environment Variable STACKQL_AUDIT_ROLE_ARN
82+
Should Not Be Empty ${awsRoleArn}
83+
${awsAuthCfg} = Catenate
84+
... { "aws": { "type": "aws_assume_role", "keyIDenvvar": "AWS_ACCESS_KEY_ID", "credentialsenvvar": "AWS_SECRET_ACCESS_KEY", "aws_role_arn": "${awsRoleArn}" } }
85+
${bucketObjectsListQuery} = Catenate
86+
... select * from aws.s3.bucket_abac where Bucket = 'stackql-trial-bucket-02' and region = 'ap-southeast-2';
87+
${result} = Run Process
88+
... ${STACKQL_EXE}
89+
... \-\-auth
90+
... ${awsAuthCfg}
91+
... \-\-registry
92+
... { "url": "file://${REPOSITORY_ROOT}/test/registry", "localDocRoot": "${REPOSITORY_ROOT}/test/registry", "verifyConfig": { "nopVerify": true } }
93+
... exec
94+
... ${bucketObjectsListQuery}
95+
... cwd=${REPOSITORY_ROOT}
96+
... stdout=${CURDIR}/tmp/AWS-S3-Bucket-Objects-List.tmp
97+
... stderr=${CURDIR}/tmp/AWS-S3-Bucket-Objects-List-stderr.tmp
98+
Should Be Equal As Integers ${result.rc} 0
99+
Should Contain ${result.stdout} stackql\-trial\-bucket\-02

‎test/robot/foreign-integration-traffic-lights/tmp/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
*
2+
!.gitignore

‎test/robot/reports-foreign-integration-traffic-lights/.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
*
2+
!.gitignore

0 commit comments

Comments
 (0)