Skip to content

Commit 436d3cd

Browse files
nohwndCopilot
nohwnd
and
Copilot
committed
Break down mock perf benchmark fix-by-fix on 5.1 and 7
Replace the binary CHANGES-vs-CLEAN benchmark with a cumulative per-fix breakdown: revert src to base (2f6ca66), then apply fix1..fix5 cumulatively, rebuilding and measuring on PS7 and Windows PowerShell 5.1 at each step (CLEAN, +FIX1, +FIX1-2, +FIX1-3, ALL5). Patches in _perf/patches isolate each fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 16292ea commit 436d3cd

7 files changed

Lines changed: 191 additions & 21 deletions

File tree

‎.github/workflows/perf-mock-benchmark.yml‎

Lines changed: 67 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,15 @@
11
name: Mock perf benchmark
22

3-
# Temporary, manually-triggered benchmark to compare the mock hot path with and
4-
# without the perf changes on Windows PowerShell 5.1 and PowerShell 7.
5-
# Runs automatically on every push to the benchmark branch (workflow_dispatch
6-
# does not work here because this file is not on the default branch).
3+
# Temporary, push-triggered benchmark to break down the mock hot-path perf changes
4+
# fix-by-fix on Windows PowerShell 5.1 and PowerShell 7. The branch HEAD already
5+
# contains all five fixes; we revert the two changed source files to the clean
6+
# pre-fix base (2f6ca66) and then re-apply individual fix hunks cumulatively,
7+
# rebuilding and re-measuring between each step. This isolates each fix's effect.
8+
#
9+
# Fixes (see _perf/patches): 1=Repair-EnumParameters early-out, 2=Test-ParameterFilter
10+
# PSVariable save/restore, 3=Set-ScriptBlockScope simple, 4=Resolve-Command simple,
11+
# 5=Test-ParameterFilter simple. workflow_dispatch only works on the default branch,
12+
# so trigger on push to this feature branch instead.
713

814
on:
915
workflow_dispatch:
@@ -13,40 +19,80 @@ on:
1319

1420
jobs:
1521
benchmark:
16-
name: Mock perf (5.1 vs 7)
22+
name: Mock perf breakdown (5.1 vs 7)
1723
runs-on: windows-latest
1824
steps:
1925
- name: Checkout
2026
uses: actions/checkout@v4
2127
with:
2228
fetch-depth: 10 # need history back to the clean base commit to revert the perf changes
2329

24-
- name: Build Pester (with perf changes)
30+
- name: Revert perf source to clean base (2f6ca66)
2531
shell: pwsh
26-
run: ./build.ps1 -Clean
32+
run: |
33+
git checkout 2f6ca66 -- src/functions/Mock.ps1 src/functions/Pester.Scoping.ps1
34+
git --no-pager diff --stat 2f6ca66 -- src/functions/Mock.ps1 src/functions/Pester.Scoping.ps1
2735
28-
- name: Benchmark WITH changes - PowerShell 7
36+
- name: Build CLEAN
2937
shell: pwsh
30-
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'CHANGES'
31-
32-
- name: Benchmark WITH changes - Windows PowerShell 5.1
38+
run: ./build.ps1 -Clean
39+
- name: CLEAN - PS7
40+
shell: pwsh
41+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'CLEAN'
42+
- name: CLEAN - 5.1
3343
shell: powershell
34-
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'CHANGES'
44+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'CLEAN'
3545

36-
- name: Revert the perf changes (clean source at the pre-fix base commit)
46+
- name: Apply Fix1
3747
shell: pwsh
38-
run: |
39-
git checkout 2f6ca66 -- src/functions/Mock.ps1 src/functions/Pester.Scoping.ps1
40-
git --no-pager diff --stat 2f6ca66 -- src/functions/Mock.ps1 src/functions/Pester.Scoping.ps1
48+
run: git apply --ignore-whitespace _perf/patches/fix1_repair.patch
49+
- name: Build +FIX1
50+
shell: pwsh
51+
run: ./build.ps1
52+
- name: '+FIX1 - PS7'
53+
shell: pwsh
54+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label '+FIX1'
55+
- name: '+FIX1 - 5.1'
56+
shell: powershell
57+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label '+FIX1'
4158

42-
- name: Rebuild Pester (clean source, PowerShell-only change)
59+
- name: Apply Fix2
60+
shell: pwsh
61+
run: git apply --ignore-whitespace _perf/patches/fix2_psvar.patch
62+
- name: Build +FIX1-2
4363
shell: pwsh
4464
run: ./build.ps1
65+
- name: '+FIX1-2 - PS7'
66+
shell: pwsh
67+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label '+FIX1-2'
68+
- name: '+FIX1-2 - 5.1'
69+
shell: powershell
70+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label '+FIX1-2'
4571

46-
- name: Benchmark CLEAN baseline - PowerShell 7
72+
- name: Apply Fix3
4773
shell: pwsh
48-
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'CLEAN'
74+
run: git apply --ignore-whitespace _perf/patches/fix3_scoping.patch
75+
- name: Build +FIX1-3
76+
shell: pwsh
77+
run: ./build.ps1
78+
- name: '+FIX1-3 - PS7'
79+
shell: pwsh
80+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label '+FIX1-3'
81+
- name: '+FIX1-3 - 5.1'
82+
shell: powershell
83+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label '+FIX1-3'
4984

50-
- name: Benchmark CLEAN baseline - Windows PowerShell 5.1
85+
- name: Apply Fix4 + Fix5 (all five)
86+
shell: pwsh
87+
run: |
88+
git apply --ignore-whitespace _perf/patches/fix4_resolve.patch
89+
git apply --ignore-whitespace _perf/patches/fix5_testfilter.patch
90+
- name: Build ALL5
91+
shell: pwsh
92+
run: ./build.ps1
93+
- name: ALL5 - PS7
94+
shell: pwsh
95+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'ALL5'
96+
- name: ALL5 - 5.1
5197
shell: powershell
52-
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'CLEAN'
98+
run: ./_perf/Measure-MockPerf.ps1 -Runs 9 -Label 'ALL5'

‎_perf/patches/.gitattributes‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
_perf/patches/*.patch text eol=lf

‎_perf/patches/fix1_repair.patch‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
diff --git a/src/functions/Mock.ps1 b/src/functions/Mock.ps1
2+
index a6876b2..c6510e8 100644
3+
--- a/src/functions/Mock.ps1
4+
+++ b/src/functions/Mock.ps1
5+
@@ -1979,6 +1981,13 @@ function Repair-EnumParameters {
6+
# broken arguments (unquoted strings) will show as NamedArguments in ast, while valid arguments are PositionalArguments.
7+
# https://github.com/pester/Pester/issues/1496
8+
# https://github.com/PowerShell/PowerShell/issues/17546
9+
+ # Fast path: parsing the param block and walking its AST is relatively expensive and runs for every
10+
+ # mock. A broken ValidateRange attribute can only exist when the param block mentions ValidateRange at
11+
+ # all, so skip the parse + AST walk entirely when it does not (the common case).
12+
+ if ($ParamBlock -notmatch 'ValidateRange') {
13+
+ return $ParamBlock
14+
+ }
15+
+
16+
$ast = [System.Management.Automation.Language.Parser]::ParseInput("param($ParamBlock)", [ref]$null, [ref]$null)
17+
$brokenValidateRange = $ast.FindAll({
18+
param($node)

‎_perf/patches/fix2_psvar.patch‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
diff --git a/src/functions/Mock.ps1 b/src/functions/Mock.ps1
2+
index a6876b2..c6510e8 100644
3+
--- a/src/functions/Mock.ps1
4+
+++ b/src/functions/Mock.ps1
5+
@@ -1231,18 +1233,18 @@ function Test-ParameterFilter {
6+
7+
$parameterFilterInvocations = [Collections.Generic.List[string]]@()
8+
9+
- $previousIsInMockParameterFilter = & $SafeCommands['Get-Variable'] -Name '______isInMockParameterFilter' -Scope Script -ValueOnly -ErrorAction Ignore
10+
+ # Save and restore the script-scoped flag with direct variable access instead of the much more
11+
+ # expensive Get-Variable/Remove-Variable cmdlets. This runs on every mock parameter-filter
12+
+ # evaluation, so the cmdlet call overhead is significant. Every reader of the script-scoped flag
13+
+ # uses a plain truthy check, so leaving the variable defined with its previous value (possibly
14+
+ # $null) when we are no longer in a filter is equivalent to removing it.
15+
+ $previousIsInMockParameterFilter = $ExecutionContext.SessionState.PSVariable.GetValue('______isInMockParameterFilter', $null)
16+
$script:______isInMockParameterFilter = $true
17+
try {
18+
$result = & $wrapper $parameters
19+
}
20+
finally {
21+
- if ($null -eq $previousIsInMockParameterFilter) {
22+
- & $SafeCommands['Remove-Variable'] -Name '______isInMockParameterFilter' -Scope Script -ErrorAction Ignore
23+
- }
24+
- else {
25+
- $script:______isInMockParameterFilter = $previousIsInMockParameterFilter
26+
- }
27+
+ $script:______isInMockParameterFilter = $previousIsInMockParameterFilter
28+
}
29+
$passed = [bool]$result
30+
if ($passed) {

‎_perf/patches/fix3_scoping.patch‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
diff --git a/src/functions/Pester.Scoping.ps1 b/src/functions/Pester.Scoping.ps1
2+
index 4f22ad2..b7ed556 100644
3+
--- a/src/functions/Pester.Scoping.ps1
4+
+++ b/src/functions/Pester.Scoping.ps1
5+
@@ -1,20 +1,21 @@
6+
function Set-ScriptBlockScope {
7+
- [CmdletBinding()]
8+
+ # This is intentionally a simple (non-advanced) function. It is called very frequently
9+
+ # (e.g. on every mock invocation), and advanced functions are noticeably more expensive to
10+
+ # invoke because of the extra parameter-binding machinery. It originally used two parameter
11+
+ # sets (FromSessionState / FromSessionStateInternal); when a SessionState is provided we
12+
+ # resolve its internal session state, otherwise the caller passed the internal session state
13+
+ # directly (which may be $null).
14+
param (
15+
- [Parameter(Mandatory = $true)]
16+
[scriptblock]
17+
$ScriptBlock,
18+
19+
- [Parameter(Mandatory = $true, ParameterSetName = 'FromSessionState')]
20+
[System.Management.Automation.SessionState]
21+
$SessionState,
22+
23+
- [Parameter(Mandatory = $true, ParameterSetName = 'FromSessionStateInternal')]
24+
- [AllowNull()]
25+
$SessionStateInternal
26+
)
27+
28+
- if ($PSCmdlet.ParameterSetName -eq 'FromSessionState') {
29+
+ if ($PSBoundParameters.ContainsKey('SessionState')) {
30+
$SessionStateInternal = $script:SessionStateInternalProperty.GetValue($SessionState, $null)
31+
}
32+

‎_perf/patches/fix4_resolve.patch‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
diff --git a/src/functions/Mock.ps1 b/src/functions/Mock.ps1
2+
index a6876b2..c6510e8 100644
3+
--- a/src/functions/Mock.ps1
4+
+++ b/src/functions/Mock.ps1
5+
@@ -600,10 +600,12 @@ function Remove-MockHook {
6+
}
7+
8+
function Resolve-Command {
9+
+ # Simple (non-advanced) function on purpose: it is on the hot mock-resolution path and advanced
10+
+ # functions are noticeably more expensive to invoke. SessionState is required and is always
11+
+ # supplied by callers.
12+
param (
13+
[string] $CommandName,
14+
[string] $ModuleName,
15+
- [Parameter(Mandatory)]
16+
[Management.Automation.SessionState] $SessionState
17+
)
18+

‎_perf/patches/fix5_testfilter.patch‎

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
diff --git a/src/functions/Mock.ps1 b/src/functions/Mock.ps1
2+
index a6876b2..c6510e8 100644
3+
--- a/src/functions/Mock.ps1
4+
+++ b/src/functions/Mock.ps1
5+
@@ -1147,9 +1149,10 @@ function Invoke-InMockScope {
6+
}
7+
8+
function Test-ParameterFilter {
9+
- [CmdletBinding()]
10+
+ # Simple (non-advanced) function on purpose: it runs on every mock parameter-filter evaluation
11+
+ # and advanced functions are noticeably more expensive to invoke. ScriptBlock and SessionState
12+
+ # are always supplied by callers.
13+
param (
14+
- [Parameter(Mandatory = $true)]
15+
[scriptblock]
16+
$ScriptBlock,
17+
18+
@@ -1162,7 +1165,6 @@ function Test-ParameterFilter {
19+
[System.Management.Automation.CommandMetadata]
20+
$Metadata,
21+
22+
- [Parameter(Mandatory)]
23+
[Management.Automation.SessionState]
24+
$SessionState,
25+

0 commit comments

Comments
 (0)