[v2] Implement CLI command benchmark harness script

[aemous]

Description of changes:

• Merge PR for the following pre-reviewed PRs
  • Create benchmark harness using a stubbed HTTP client #9239
  • [v2] Benchmark harness improvements & add more service benchmarks #9264

No changes apart from those in the reviewed PRs linked above are included in this PR.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[ashovlin]

Attempting to run these for me is resulting in

(.venv312) ➜  performance git:(benchmarks) ✗ ./run-benchmarks
Traceback (most recent call last):
  File "/Users/shovlia/GitHub/aws-cli/scripts/performance/./run-benchmarks", line 38, in <module>
    harness.run_benchmarks(parser.parse_args())
  File "/Users/shovlia/GitHub/aws-cli/scripts/performance/benchmark_utils.py", line 410, in run_benchmarks
    measurements = self._run_isolated_benchmark(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shovlia/GitHub/aws-cli/scripts/performance/benchmark_utils.py", line 351, in _run_isolated_benchmark
    with open(child_output_path, 'w') as out, open(child_err_path, 'w') as err:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'results/assets/output.txt'
Traceback (most recent call last):
  File "/Users/shovlia/GitHub/aws-cli/scripts/performance/./run-benchmarks", line 38, in <module>
    harness.run_benchmarks(parser.parse_args())
  File "/Users/shovlia/GitHub/aws-cli/scripts/performance/benchmark_utils.py", line 410, in run_benchmarks
    measurements = self._run_isolated_benchmark(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shovlia/GitHub/aws-cli/scripts/performance/benchmark_utils.py", line 366, in _run_isolated_benchmark
    raise RuntimeError('Child process execution failed: output file not found.')
RuntimeError: Child process execution failed: output file not found.

[ashovlin]

It looks like you're missing the changes from #9264 somehow? I only see the S3 tests in benchmarks.json, don't see the file_literals support, etc.

[aemous]

It looks like you're missing the changes from #9264 somehow? I only see the S3 tests in benchmarks.json, don't see the file_literals support, etc.

Fixed in latest revision, thanks