diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 3842b4d..f79fdbe 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml @@ -3,6 +3,8 @@ on: push: branches: - master + paths-ignore: + - README.md permissions: deployments: write diff --git a/README.md b/README.md index 99f51d8..8a139eb 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,42 @@ # Material Modules Benchmark -Generated charts are displayed **[here](https://shader-slang.com/slang-material-modules-benchmark/dev/bench/)**. +[**See the charts here**](https://shader-slang.com/slang-material-modules-benchmark/dev/bench) + +**How it works:** +1. Commits to the master branch of [Slang](https://github.com/shader-slang/slang) will trigger a CI workflow that [runs the benchmark](https://github.com/shader-slang/slang/actions/workflows/push-benchmark-results.yml) and uploads the JSON file to **this** repository (specifically to `benchmarks.json`). +2. Changes to `benchmark.json` will trigger a CI workflow in this repository. +3. This workflow updates the `gh-pages` repository using [`github-action-benchmark`](fffffhttps://github.com/benchmark-action/github-action-benchmark). It reads the new results from `benchmarks.json` and updates a database in `gh-pages`. +4. Once `gh-pages` receives the update, it will trigger the final CI workflow that builds the GitHub Pages site. + +The diagram below summarizes these steps. + +![smmb graphs](media/smmb-graphs.png) + +*Notes:* +- In (1), the worflow from the Slang repository overrides `benchmarks.json` when it pushes to this repository; this is expected to happen. +- This repository contains another file, `currrent`, which holds the latest commit's message and hash for debugging purposes. +- Each time `benchmarks.json` is updated (2), the `github-action-benchmark` workflow reads its contents and appends a database that is embedded in a [Javascript file](https://github.com/shader-slang/slang-material-modules-benchmark/blob/gh-pages/dev/bench/data.js). +- There is currently no behaviour to limit the number of entries in the database, so it can grow to a couple megabytes. Manually trimming the data is possible by directly editing the Javascript file and removing entries. + +**How to read the charts:** +- Each chart is title with the format ` : : ` + - `` is one of `closesthit`, `anyhit`, and `shadow` + - `` is either `mono` for monolithic compilation or `module` for modular compilation. + - `` is currently fixed to DXIL. + - Other targets can be generated by running the benchmarking script with a different target (DXIL or SPIRV; with or without precompiled modules). +- The $x$-axis tracks the commit hash. Unfortunately there is currently no way to display concrete dates. +- The $y$-axis shows the time, in milliseconds, taken to compile the specific shader stage under the particular compilation mode and target. + +**Interacting with the charts:** + +In case there is a commit which results in alarming measurements, there is a convenient way to reach the original commit/PR. Each data point of each graph can be highlighted as so: + +![nodes](media/nodes.png) + +Clicking on the node will result in a redirection to the associated commit in **this** repository. + +![nodes](media/one.png) + +Clicking on the highlighted link will then redirect the user to original commit/PR in the Slang repository. + +![nodes](media/two.png) \ No newline at end of file diff --git a/media/nodes.png b/media/nodes.png new file mode 100644 index 0000000..0c5e17c Binary files /dev/null and b/media/nodes.png differ diff --git a/media/one.png b/media/one.png new file mode 100644 index 0000000..5fb2fac Binary files /dev/null and b/media/one.png differ diff --git a/media/smmb-graphs.png b/media/smmb-graphs.png new file mode 100644 index 0000000..d0e8c02 Binary files /dev/null and b/media/smmb-graphs.png differ diff --git a/media/two.png b/media/two.png new file mode 100644 index 0000000..25839ca Binary files /dev/null and b/media/two.png differ