Skip to content

Commit 8a3d6ca

Browse files
kamyarz-awskamyarz-aws
authored
Update to Version v3.3.0 (#213)
Update to Version v3.3.0
1 parent 274cfac commit 8a3d6ca

File tree

101 files changed

+32897
-30142
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

101 files changed

+32897
-30142
lines changed

‎CHANGELOG.md

+30
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,36 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [3.3.0] - 2024-09-16
9+
10+
### Added
11+
12+
- Added cron scheduling (https://github.com/aws-solutions/distributed-load-testing-on-aws/issues/84)
13+
- Added jtl report to s3 bucket logs (https://github.com/aws-solutions/distributed-load-testing-on-aws/issues/150)
14+
- Added enhanced mechanism to collect Anonymized Metrics Feature
15+
- Added new integration tests
16+
17+
### Fixed
18+
19+
- Fixed issues pertaining to (https://github.com/aws-solutions/distributed-load-testing-on-aws/issues/193)
20+
21+
### Security
22+
23+
- `micromatch` to mitigate [CVE-2024-4067](https://github.com/advisories/GHSA-952p-6rrq-rcjv)
24+
- `webpack` to mitigate [CVE-2024-43788](https://github.com/advisories/GHSA-4vvj-4cpr-p986)
25+
- `path-to-regexp` to mitigate [CVE-2024-45296](https://github.com/advisories/GHSA-9wv6-86v2-598j)
26+
- `serve-static` to mitigate [CVE-2024-43800](https://github.com/advisories/GHSA-cm22-4g7w-348p)
27+
- `body-parser` to mitigate [CVE-2024-45590](https://github.com/advisories/GHSA-qwcr-r2fm-qrc7)
28+
- `express` to mitigate [CVE-2024-43796](https://github.com/advisories/GHSA-qw6h-vgh9-j6wx)
29+
- `send` to mitigate [CVE-2024-43799](https://github.com/advisories/GHSA-m6fv-jmcg-4jfg)
30+
- `Flask_Cors` on Docker image to mitigate [CVE-2024-6221](https://github.com/advisories/GHSA-hxwh-jpp2-84pm)
31+
32+
### Updated
33+
34+
- Updated aws-cdk to 2.155.0
35+
- Updated aws-cdk-lib to 2.155.0
36+
- Updated @aws-cdk/aws-servicecatalogappregistry-alpha to 2.155.0-alpha.0
37+
838
## [3.2.11] - 2024-08-19
939

1040
### Updated

‎NOTICE

+3-1
Original file line numberDiff line numberDiff line change
@@ -66,4 +66,6 @@ ts-node under the Massachusetts Institute of Technology (MIT) License
6666
typescript under the Apache License 2.0
6767
uuid under the Massachusetts Institute of Technology (MIT) license
6868
xml-js under the Massachusetts Institute of Technology (MIT) license
69-
source-map-support under the Massachusetts Institute of Technology (MIT) license
69+
source-map-support under the Massachusetts Institute of Technology (MIT) license
70+
react-js-cron under the Massachusetts Institute of Technology (MIT) license
71+
cron-parser under the Massachusetts Institute of Technology (MIT) license

‎README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ The Distributed Load Testing Solution leverages managed, highly available and hi
1515

1616
## Deployment
1717

18-
The solution is deployed using a CloudFormation template with a lambda backed custom resource. To simulate users from regions other than the region the solution is initially deployed in, a regional template must be deployed within the other desired regions. For details on deploying the solution please see the details on the solution implementation guide: [Distributed Load Testing](https://docs.aws.amazon.com/solutions/latest/distributed-load-testing-on-aws/deployment.html)
18+
The solution is deployed using a CloudFormation template with a lambda backed custom resource. To simulate users from regions other than the region the solution is initially deployed in, a regional template must be deployed within the other desired regions. For details on deploying the solution please see the details on the solution implementation guide: [Distributed Load Testing](https://docs.aws.amazon.com/solutions/latest/distributed-load-testing-on-aws/deployment.html).
1919

2020
## Source Code
2121

‎VERSION.txt

+1-1
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
3.2.11
1+
3.3.0

‎deployment/build-s3-dist.sh

+26
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,32 @@ manifest=(`find * -type f ! -iname ".DS_Store"`)
162162
manifest_json=$(IFS=,;printf "%s" "${manifest[*]}")
163163
echo "[\"${manifest_json}\"]" | sed 's/,/","/g' > ./console-manifest.json
164164

165+
echo "------------------------------------------------------------------------------"
166+
echo "Build Metrics utils lambda archive."
167+
echo "------------------------------------------------------------------------------"
168+
cd ${source_dir}/metrics-utils
169+
npm run build && npm run zip
170+
if [ $? -eq 0 ]
171+
then
172+
echo "Build for metrics-utils succeeded"
173+
else
174+
echo "******************************************************************************"
175+
echo "Build FAILED for metrics-utils"
176+
echo "******************************************************************************"
177+
exit 1
178+
fi
179+
mv ${source_dir}/metrics-utils/dist/*.zip ${build_dist_dir}/metrics-utils.zip
180+
if [ $? -eq 0 ]
181+
then
182+
echo "moved metrics-utils.zip to ${build_dist_dir}"
183+
else
184+
echo "******************************************************************************"
185+
echo "FAILED to move metrics-utils.zip to ${build_dist_dir}"
186+
echo "******************************************************************************"
187+
exit 1
188+
fi
189+
190+
165191
echo "------------------------------------------------------------------------------"
166192
echo "Build S3 Packaging Complete"
167193
echo "------------------------------------------------------------------------------"

‎deployment/ecr/distributed-load-testing-on-aws-load-tester/Dockerfile

+4-4
Original file line numberDiff line numberDiff line change
@@ -48,10 +48,10 @@ RUN rm -rf /usr/share/dotnet
4848
RUN rm -rf /usr/local/lib/python3.10/dist-packages/aiohttp*
4949
RUN pip install --upgrade aiohttp
5050

51-
# Replacing idna with more stable version to resolve CVE-2024-3651
52-
RUN pip install --upgrade idna
53-
RUN rm -rf /root/.bzt/python-packages/3.10.12/idna*
54-
RUN cp -r /usr/local/lib/python3.10/dist-packages/idna* /root/.bzt/python-packages/3.10.12/
51+
# Replacing idna and Flas_Cors with more stable version to resolve CVE-2024-3651 and CVE-2024-6221
52+
RUN pip install --upgrade idna Flask_Cors
53+
RUN rm -rf /root/.bzt/python-packages/3.10.12/idna* /root/.bzt/python-packages/3.10.12/Flask_Cors*
54+
RUN cp -r /usr/local/lib/python3.10/dist-packages/idna* /usr/local/lib/python3.10/dist-packages/Flask_Cors* /root/.bzt/python-packages/3.10.12/
5555

5656
WORKDIR /bzt-configs/
5757
ENTRYPOINT ["./load-test.sh"]

‎deployment/ecr/distributed-load-testing-on-aws-load-tester/load-test.sh

+77-31
Original file line numberDiff line numberDiff line change
@@ -25,26 +25,46 @@ trap 'sigterm_handler' SIGTERM
2525
echo "Download test scenario"
2626
aws s3 cp s3://$S3_BUCKET/test-scenarios/$TEST_ID-$AWS_REGION.json test.json --region $MAIN_STACK_REGION
2727

28+
# Set the default log file values to jmeter
29+
LOG_FILE="jmeter.log"
30+
OUT_FILE="jmeter.out"
31+
ERR_FILE="jmeter.err"
32+
KPI_EXT="jtl"
33+
2834
# download JMeter jmx file
2935
if [ "$TEST_TYPE" != "simple" ]; then
30-
# Copy *.jar to JMeter library path. See the Taurus JMeter path: https://gettaurus.org/docs/JMeter/
31-
JMETER_LIB_PATH=`find ~/.bzt/jmeter-taurus -type d -name "lib"`
32-
echo "cp $PWD/*.jar $JMETER_LIB_PATH"
33-
cp $PWD/*.jar $JMETER_LIB_PATH
36+
# setting the log file values to the test type
37+
LOG_FILE="${TEST_TYPE}.log"
38+
OUT_FILE="${TEST_TYPE}.out"
39+
ERR_FILE="${TEST_TYPE}.err"
40+
41+
# set variables based on TEST_TYPE
42+
if [ "$TEST_TYPE" == "jmeter" ]; then
43+
EXT="jmx"
44+
TYPE_NAME="JMeter"
45+
# Copy *.jar to JMeter library path. See the Taurus JMeter path: https://gettaurus.org/docs/JMeter/
46+
JMETER_LIB_PATH=`find ~/.bzt/jmeter-taurus -type d -name "lib"`
47+
echo "cp $PWD/*.jar $JMETER_LIB_PATH"
48+
cp $PWD/*.jar $JMETER_LIB_PATH
49+
50+
fi
3451

3552
if [ "$FILE_TYPE" != "zip" ]; then
36-
aws s3 cp s3://$S3_BUCKET/public/test-scenarios/$TEST_TYPE/$TEST_ID.jmx ./ --region $MAIN_STACK_REGION
53+
aws s3 cp s3://$S3_BUCKET/public/test-scenarios/$TEST_TYPE/$TEST_ID.$EXT ./ --region $MAIN_STACK_REGION
3754
else
3855
aws s3 cp s3://$S3_BUCKET/public/test-scenarios/$TEST_TYPE/$TEST_ID.zip ./ --region $MAIN_STACK_REGION
3956
unzip $TEST_ID.zip
40-
# only looks for the first jmx file.
41-
JMETER_SCRIPT=`find . -name "*.jmx" | head -n 1`
42-
if [ -z "$JMETER_SCRIPT" ]; then
43-
echo "There is no JMeter script in the zip file."
57+
echo "UNZIPPED"
58+
ls -l
59+
# only looks for the first test script file.
60+
TEST_SCRIPT=`find . -name "*.${EXT}" | head -n 1`
61+
echo $TEST_SCRIPT
62+
if [ -z "$TEST_SCRIPT" ]; then
63+
echo "There is no test script (.${EXT}) in the zip file."
4464
exit 1
4565
fi
4666

47-
sed -i -e "s|$TEST_ID.jmx|$JMETER_SCRIPT|g" test.json
67+
sed -i -e "s|$TEST_ID.$EXT|$TEST_SCRIPT|g" test.json
4868

4969
# copy bundled plugin jars to jmeter extension folder to make them available to jmeter
5070
BUNDLED_PLUGIN_DIR=`find $PWD -type d -name "plugins" | head -n 1`
@@ -75,34 +95,54 @@ else
7595
fi
7696

7797
echo "Running test"
98+
7899
stdbuf -i0 -o0 -e0 bzt test.json -o modules.console.disable=true | stdbuf -i0 -o0 -e0 tee -a result.tmp | sed -u -e "s|^|$TEST_ID $LIVE_DATA_ENABLED |"
79100
CALCULATED_DURATION=`cat result.tmp | grep -m1 "Test duration" | awk -F ' ' '{ print $5 }' | awk -F ':' '{ print ($1 * 3600) + ($2 * 60) + $3 }'`
80101

81102
# upload custom results to S3 if any
82103
# every file goes under $TEST_ID/$PREFIX/$UUID to distinguish the result correctly
83104
if [ "$TEST_TYPE" != "simple" ]; then
84105
if [ "$FILE_TYPE" != "zip" ]; then
85-
cat $TEST_ID.jmx | grep filename > results.txt
106+
cat $TEST_ID.$EXT | grep filename > results.txt
86107
else
87-
cat $JMETER_SCRIPT | grep filename > results.txt
108+
cat $TEST_SCRIPT | grep filename > results.txt
88109
fi
89-
sed -i -e 's/<stringProp name="filename">//g' results.txt
90-
sed -i -e 's/<\/stringProp>//g' results.txt
91-
sed -i -e 's/ //g' results.txt
92-
93-
echo "Files to upload as results"
94-
cat results.txt
95-
96-
files=(`cat results.txt`)
97-
for f in "${files[@]}"; do
98-
p="s3://$S3_BUCKET/results/$TEST_ID/JMeter_Result/$PREFIX/$UUID/$f"
99-
if [[ $f = /* ]]; then
100-
p="s3://$S3_BUCKET/results/$TEST_ID/JMeter_Result/$PREFIX/$UUID$f"
101-
fi
102110

103-
echo "Uploading $p"
104-
aws s3 cp $f $p --region $MAIN_STACK_REGION
105-
done
111+
if [ -f results.txt ]; then
112+
sed -i -e 's/<stringProp name="filename">//g' results.txt
113+
sed -i -e 's/<\/stringProp>//g' results.txt
114+
sed -i -e 's/ //g' results.txt
115+
116+
echo "Files to upload as results"
117+
cat results.txt
118+
119+
files=(`cat results.txt`)
120+
extensions=()
121+
for f in "${files[@]}"; do
122+
ext="${f##*.}"
123+
if [[ ! " ${extensions[@]} " =~ " ${ext} " ]]; then
124+
extensions+=("$ext")
125+
fi
126+
done
127+
128+
# Find all files in the current folder with the same extensions
129+
all_files=()
130+
for ext in "${extensions[@]}"; do
131+
for f in *."$ext"; do
132+
all_files+=("$f")
133+
done
134+
done
135+
136+
for f in "${all_files[@]}"; do
137+
p="s3://$S3_BUCKET/results/$TEST_ID/${TYPE_NAME}_Result/$PREFIX/$UUID/$f"
138+
if [[ $f = /* ]]; then
139+
p="s3://$S3_BUCKET/results/$TEST_ID/${TYPE_NAME}_Result/$PREFIX/$UUID$f"
140+
fi
141+
142+
echo "Uploading $p"
143+
aws s3 cp $f $p --region $MAIN_STACK_REGION
144+
done
145+
fi
106146
fi
107147

108148
if [ -f /tmp/artifacts/results.xml ]; then
@@ -114,12 +154,18 @@ if [ -f /tmp/artifacts/results.xml ]; then
114154
xmlstarlet ed -L -u /FinalStatus/TestDuration -v $CALCULATED_DURATION /tmp/artifacts/results.xml
115155
fi
116156

157+
if [ "$TEST_TYPE" == "simple" ]; then
158+
TEST_TYPE = "jmeter"
159+
fi
160+
117161
echo "Uploading results, bzt log, and JMeter log, out, and err files"
118162
aws s3 cp /tmp/artifacts/results.xml s3://$S3_BUCKET/results/${TEST_ID}/${PREFIX}-${UUID}-${AWS_REGION}.xml --region $MAIN_STACK_REGION
119163
aws s3 cp /tmp/artifacts/bzt.log s3://$S3_BUCKET/results/${TEST_ID}/bzt-${PREFIX}-${UUID}-${AWS_REGION}.log --region $MAIN_STACK_REGION
120-
aws s3 cp /tmp/artifacts/jmeter.log s3://$S3_BUCKET/results/${TEST_ID}/jmeter-${PREFIX}-${UUID}-${AWS_REGION}.log --region $MAIN_STACK_REGION
121-
aws s3 cp /tmp/artifacts/jmeter.out s3://$S3_BUCKET/results/${TEST_ID}/jmeter-${PREFIX}-${UUID}-${AWS_REGION}.out --region $MAIN_STACK_REGION
122-
aws s3 cp /tmp/artifacts/jmeter.err s3://$S3_BUCKET/results/${TEST_ID}/jmeter-${PREFIX}-${UUID}-${AWS_REGION}.err --region $MAIN_STACK_REGION
164+
aws s3 cp /tmp/artifacts/$LOG_FILE s3://$S3_BUCKET/results/${TEST_ID}/${TEST_TYPE}-${PREFIX}-${UUID}-${AWS_REGION}.log --region $MAIN_STACK_REGION
165+
aws s3 cp /tmp/artifacts/$OUT_FILE s3://$S3_BUCKET/results/${TEST_ID}/${TEST_TYPE}-${PREFIX}-${UUID}-${AWS_REGION}.out --region $MAIN_STACK_REGION
166+
aws s3 cp /tmp/artifacts/$ERR_FILE s3://$S3_BUCKET/results/${TEST_ID}/${TEST_TYPE}-${PREFIX}-${UUID}-${AWS_REGION}.err --region $MAIN_STACK_REGION
167+
aws s3 cp /tmp/artifacts/kpi.${KPI_EXT} s3://$S3_BUCKET/results/${TEST_ID}/kpi-${PREFIX}-${UUID}-${AWS_REGION}.${KPI_EXT} --region $MAIN_STACK_REGION
168+
123169
else
124170
echo "An error occurred while the test was running."
125171
fi

‎source/.eslintrc

+16-6
Original file line numberDiff line numberDiff line change
@@ -4,16 +4,17 @@
44
"env": {
55
"jest": true,
66
"node": true,
7-
"es6": true
7+
"es6": true,
8+
"cypress/globals": true
89
},
9-
"ignorePatterns": ["**/node_modules/**"],
10+
"ignorePatterns": ["**/node_modules/**", "**/*.config.ts", "**/dist/**", "**/cdk.out/**"],
1011
"parserOptions": {
1112
"ecmaVersion": "latest"
1213
},
1314
"overrides": [
1415
{
1516
"files": ["**/*.ts", "**/*.tsx"],
16-
"plugins": ["@typescript-eslint", "import", "header"],
17+
"plugins": ["@typescript-eslint", "import", "header", "cypress"],
1718
"extends": [
1819
"eslint:recommended",
1920
"plugin:@typescript-eslint/recommended",
@@ -32,8 +33,12 @@
3233
"rules": {
3334
"@typescript-eslint/no-inferrable-types": ["off", { "ignoreParameters": true, "ignoreProperties": true }],
3435
"@typescript-eslint/no-useless-constructor": ["off"],
35-
"@typescript-eslint/no-unused-vars": ["error", { "args": "none", "argsIgnorePattern": "^_", "varsIgnorePattern": "^_" }],
36+
"@typescript-eslint/no-unused-vars": [
37+
"error",
38+
{ "args": "none", "argsIgnorePattern": "^_", "varsIgnorePattern": "^_" }
39+
],
3640
"@typescript-eslint/no-throw-literal": ["error"],
41+
"@typescript-eslint/no-empty-interface": ["off"],
3742
"no-new": 0
3843
}
3944
},
@@ -51,12 +56,17 @@
5156
"no-useless-constructor": ["off"],
5257
"no-throw-literal": ["off"],
5358

54-
"header/header": ["error", "line", [" Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.", " SPDX-License-Identifier: Apache-2.0"], 2],
59+
"header/header": [
60+
"error",
61+
"line",
62+
[" Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.", " SPDX-License-Identifier: Apache-2.0"],
63+
2
64+
],
5565

5666
"jsdoc/require-param-type": ["off"],
5767
"jsdoc/require-returns-type": ["off"],
5868
"jsdoc/newline-after-description": ["off"],
5969

6070
"import/no-unresolved": 1 // warn only on Unable to resolve path import/no-unresolved
6171
}
62-
}
72+
}

‎source/api-services/index.js

+5-1
Original file line numberDiff line numberDiff line change
@@ -48,13 +48,17 @@ class APIHandler {
4848
);
4949
}
5050
// Handle creating test
51-
else data = await scenarios.createTest(config);
51+
else data = await scenarios.createTest(config, functionName);
5252
if (process.env.SEND_METRIC === "Yes") {
5353
await utils.sendMetric({
5454
Type: "TaskCreate",
5555
TestType: config.testType,
5656
FileType: config.fileType || (config.testType === "simple" ? "none" : "script"),
5757
TaskCount: config.taskCount,
58+
TestId: data.testId,
59+
TestScheduleStep: config.scheduleStep,
60+
CronValue: config.cronValue,
61+
TestEventBridgeScheduled: config.eventBridge,
5862
});
5963
}
6064
return data;

0 commit comments

Comments
 (0)