chore: integration: add tests for browser scripts and metrics #81
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
nadiamoe
changed the title
nadiamoe
requested review from
d0ugal,
a team and
The-9880
and removed request for
a team
nadiamoe
force-pushed
the
integration-browser-2
branch
from
1b1763c to
3830589
Compare
d0ugal
approved these changes
Mar 11, 2025
|
|
||
| try { | ||
| // e-commerce site as a torture test for metric generation. | ||
| await page.goto('https://www.amazon.com'); |
There was a problem hiding this comment.
Rather than depending on an external source, we could use custom metrics in k6 but with the builtin names. This would likely increase the stability but it a little tedious. An idea for the future maybe!
There was a problem hiding this comment.
Yep, I think producing a torture-test locally would be a good next step!
Ideally we should return some actual HTML with loads of (mocked) images and scripts.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a browser script to the integration test suite, which for now only tests for presence of a couple trivial metrics.
Now, of course that makes the thing look easier than it is: To run browser tests we need a browser, and to run browser tests on CI/CD we need to either be reckless and install chromium on the GitHub runners, or to use Crocochrome. As I think we all like the second alternative better, this PR introduces Crocochrome.
Even though the croc simplifies things a lot, there are a couple things that tests need to take care of:
In https://github.com/grafana/sm-k6-runner/pull/201/, which did a very similar thing for the runner, I chose to use testcontainers for crocochrome, mainly because the testcontainers dep was already there. Here, I'm trying to see how crazy it would be to not use testcontainers, and instead YOLO-run
dockercommands.Looking forward to reviewer's thoughts on how that looks: If someone strongly opposes to trying this as an experiment, I can fall back to testcontainers and its gazillion dependencies. If there's no strong opposition, I suggest we give this a try: Falling back to testcontainers if this is flaky should be close to no effort.
As for the crocochrome API, a couple of quick & dirty functions are implemented to use it.