You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We were using globby.stream, which gives no guarantees on the order of the files returned by the stream. Instead of relying on the async api here we can use the synchronous api and sort the resulting files to get a more deterministic order as the files are changing.
I added a test, but I skip it by default as it has to run many iterations to maybe fail
Notes for release
TypeGen: Sort the scanned files to get a more consistent order of the generated types. Note that first time running this after upgrade you might get a new result because of an earlier mismatch
efps — editor "frames per second". The number of updates assumed to be possible within a second.
Derived from input latency. efps = 1000 / input_latency
Detailed information
🏠 Reference result
The performance result of sanity@latest
Benchmark
latency
p75
p90
p99
blocking time
test duration
article (title)
41ms
47ms
55ms
423ms
436ms
11.7s
article (body)
14ms
16ms
30ms
78ms
254ms
5.7s
article (string inside object)
36ms
39ms
41ms
137ms
118ms
6.6s
article (string inside array)
41ms
43ms
48ms
148ms
136ms
6.8s
recipe (name)
22ms
24ms
26ms
47ms
0ms
7.6s
recipe (description)
19ms
21ms
24ms
61ms
0ms
4.7s
recipe (instructions)
7ms
8ms
9ms
49ms
0ms
3.4s
synthetic (title)
50ms
52ms
56ms
210ms
560ms
12.1s
synthetic (string inside object)
49ms
52ms
76ms
556ms
1274ms
8.6s
🧪 Experiment result
The performance result of this branch
Benchmark
latency
p75
p90
p99
blocking time
test duration
article (title)
45ms
50ms
87ms
467ms
1083ms
12.8s
article (body)
14ms
17ms
28ms
248ms
368ms
6.0s
article (string inside object)
43ms
44ms
54ms
162ms
322ms
7.7s
article (string inside array)
46ms
51ms
55ms
147ms
382ms
7.8s
recipe (name)
21ms
22ms
25ms
35ms
27ms
7.1s
recipe (description)
19ms
20ms
22ms
44ms
0ms
4.6s
recipe (instructions)
6ms
8ms
9ms
38ms
0ms
3.5s
synthetic (title)
50ms
52ms
58ms
219ms
794ms
12.9s
synthetic (string inside object)
48ms
50ms
55ms
409ms
841ms
7.8s
📚 Glossary
column definitions
benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
test duration — how long the test run took to complete.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
We were using globby.stream, which gives no guarantees on the order of the files returned by the stream. Instead of relying on the async api here we can use the synchronous api and sort the resulting files to get a more deterministic order as the files are changing.
Alternative to #8598
What to review
Correctness
Testing
I added a test, but I skip it by default as it has to run many iterations to maybe fail
Notes for release
TypeGen: Sort the scanned files to get a more consistent order of the generated types. Note that first time running this after upgrade you might get a new result because of an earlier mismatch