Skip to content

Commit 6bd6225

Browse files
committed
fix up readme
1 parent ffed1a8 commit 6bd6225

File tree

1 file changed

+55
-0
lines changed

1 file changed

+55
-0
lines changed

README.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Package Manager Benchmarks
2+
3+
This repo contains a suite of fixutres & tools to track the performance of package managers.
4+
5+
### Environment
6+
7+
We current only test the latest `linux` runner which is the most common GitHub Action environment. The [standard GitHub-hosted public runner environment](https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories) specs are below:
8+
9+
- VM: Linux
10+
- Processor (CPU): 4
11+
- Memory (RAM): 16 GB
12+
- Storage (SSD): 14 GB
13+
- Workflow label: `ubuntu-latest`
14+
15+
We may add Mac/Windows in the future but the it will likely exponentially increase the already slow run time of the suite (**~1hr**).
16+
17+
### Configuration/Normalization
18+
19+
We do a best-effort job to configure each tool to behave as similar as possible to its peers but there's limitations to this standardization in many scenarios (as each tool makes decisions about its default support for security checks/validations/feature-set). As part of the normalization process, we count the number of packages - post-installation - & use that to determine the average speed relative to the number of packages installed. This strategy helps account for when there are significant discrepencies between the package manager's dependency graph resolution ([you can read/see more here](https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing)).
20+
21+
#### Example:
22+
23+
- **Package Manager A** installs **1,000** packages in **10s** -> an avg. of **~10ms** per-package
24+
- **Package Manager B** installs **10** packages in **1s** -> an avg. of **~100ms** per-package
25+
26+
### Testing Package Installation
27+
28+
The installation tests we run today mimic a cold-cache scenario for a variety of test fixtures (ie. we install the packages of a `next`, `vue`, `svelte` & `astro` starter project). We will likely add lockfile & warm cache tests in the near future.
29+
30+
#### Supported Tools
31+
32+
- `vlt`
33+
- `npm`
34+
- `pnpm`
35+
- `yarn`
36+
- `yarn berry`
37+
- `deno`
38+
- `bun`
39+
40+
### Testing Script Execution
41+
42+
This suite also tests the performance of basic script execution (ex. `npm run foo`). Notably, for any given build, test or deployment task the spawning of the process is a fraction of the overall execution time. That said, this is a commonly tracked workflow by various developer tools as it involves the common set of tasks: startup, filesystem read (`package.json`) & finally, spawning the process/command.
43+
44+
#### Supported Tools
45+
46+
- `vlt`
47+
- `npm`
48+
- `pnpm`
49+
- `yarn`
50+
- `yarn berry`
51+
- `deno`
52+
- `bun`
53+
- `node`
54+
- `turborepo`
55+
- `nx`

0 commit comments

Comments
 (0)