-
Notifications
You must be signed in to change notification settings - Fork 474
fix(ci_visibility): fix corner case in Python 3.9 code coverage instrumentation [backport 4.1] #15704
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: 4.1
Are you sure you want to change the base?
Conversation
|
|
Bootstrap import analysisComparison of import times between this PR and base. SummaryThe average import time from this PR is: 248 ± 2 ms. The average import time from base is: 249 ± 2 ms. The import time difference between this PR and base is: -1.0 ± 0.1 ms. Import time breakdownThe following import paths have shrunk:
|
…umentation (#15697) ## Description In Python 3.9, `dis.findlinestarts()` (used to determine where to insert instrumentation calls) can return imprecise line number values in some corner cases. This PR ensures that we do not explode in such cases (even though the line number information may not be 100% precise). ## Testing Unit tests. ## Risks None. ## Additional Notes Perhaps a better solution would be not to rely on `dis.findlinestarts()` and implement our own routine for identifying line starts in Python 3.9. But I don't think it's worth doing given that Python 3.9 is itself past its end of life. (cherry picked from commit fbc2b78)
0460a1f to
d927fc4
Compare
Performance SLOsComparing candidate backport-15697-to-4.1 (c20c572) with baseline 4.1 (3e147cb) 📈 Performance Regressions (3 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 18.098µs (SLO: <20.000µs -9.5%) vs baseline: 📈 +20.9% Memory: ✅ 42.998MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +4.9% ✅ add_inplace_aspectTime: ✅ 14.913µs (SLO: <20.000µs 📉 -25.4%) vs baseline: -0.4% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +5.0% ✅ add_inplace_noaspectTime: ✅ 0.339µs (SLO: <10.000µs 📉 -96.6%) vs baseline: +0.6% Memory: ✅ 43.057MB (SLO: <43.000MB +0.1%) vs baseline: +5.2% ✅ add_noaspectTime: ✅ 0.545µs (SLO: <10.000µs 📉 -94.6%) vs baseline: -0.9% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.9% ✅ bytearray_aspectTime: ✅ 17.974µs (SLO: <30.000µs 📉 -40.1%) vs baseline: +0.3% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.9% ✅ bytearray_extend_aspectTime: ✅ 23.701µs (SLO: <30.000µs 📉 -21.0%) vs baseline: ~same Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.9% ✅ bytearray_extend_noaspectTime: ✅ 2.651µs (SLO: <10.000µs 📉 -73.5%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% ✅ bytearray_noaspectTime: ✅ 1.447µs (SLO: <10.000µs 📉 -85.5%) vs baseline: -0.4% Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.1% ✅ bytes_aspectTime: ✅ 16.657µs (SLO: <20.000µs 📉 -16.7%) vs baseline: +0.3% Memory: ✅ 43.037MB (SLO: <43.000MB ~same) vs baseline: +5.1% ✅ bytes_noaspectTime: ✅ 1.423µs (SLO: <10.000µs 📉 -85.8%) vs baseline: +3.1% Memory: ✅ 42.703MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.1% ✅ bytesio_aspectTime: ✅ 55.254µs (SLO: <70.000µs 📉 -21.1%) vs baseline: +0.2% Memory: ✅ 43.057MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.0% ✅ bytesio_noaspectTime: ✅ 3.169µs (SLO: <10.000µs 📉 -68.3%) vs baseline: -1.4% Memory: ✅ 42.979MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ capitalize_aspectTime: ✅ 14.647µs (SLO: <20.000µs 📉 -26.8%) vs baseline: ~same Memory: ✅ 43.018MB (SLO: <43.000MB ~same) vs baseline: +4.7% ✅ capitalize_noaspectTime: ✅ 2.543µs (SLO: <10.000µs 📉 -74.6%) vs baseline: -1.6% Memory: ✅ 42.920MB (SLO: <43.000MB 🟡 -0.2%) vs baseline: +4.7% ✅ casefold_aspectTime: ✅ 14.690µs (SLO: <20.000µs 📉 -26.5%) vs baseline: +0.7% Memory: ✅ 42.979MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.9% ✅ casefold_noaspectTime: ✅ 3.082µs (SLO: <10.000µs 📉 -69.2%) vs baseline: +0.1% Memory: ✅ 42.959MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.6% ✅ decode_aspectTime: ✅ 15.440µs (SLO: <30.000µs 📉 -48.5%) vs baseline: -1.0% Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +5.0% ✅ decode_noaspectTime: ✅ 1.592µs (SLO: <10.000µs 📉 -84.1%) vs baseline: +0.4% Memory: ✅ 43.096MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +5.0% ✅ encode_aspectTime: ✅ 18.210µs (SLO: <30.000µs 📉 -39.3%) vs baseline: 📈 +22.3% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.7% ✅ encode_noaspectTime: ✅ 1.464µs (SLO: <10.000µs 📉 -85.4%) vs baseline: +0.4% Memory: ✅ 42.821MB (SLO: <43.000MB 🟡 -0.4%) vs baseline: +4.5% ✅ format_aspectTime: ✅ 171.094µs (SLO: <200.000µs 📉 -14.5%) vs baseline: ~same Memory: ✅ 43.116MB (SLO: <43.250MB 🟡 -0.3%) vs baseline: +4.8% ✅ format_map_aspectTime: ✅ 191.322µs (SLO: <200.000µs -4.3%) vs baseline: -0.2% Memory: ✅ 43.037MB (SLO: <43.500MB 🟡 -1.1%) vs baseline: +4.7% ✅ format_map_noaspectTime: ✅ 3.743µs (SLO: <10.000µs 📉 -62.6%) vs baseline: +0.4% Memory: ✅ 42.821MB (SLO: <43.250MB 🟡 -1.0%) vs baseline: +4.3% ✅ format_noaspectTime: ✅ 3.174µs (SLO: <10.000µs 📉 -68.3%) vs baseline: +0.5% Memory: ✅ 42.979MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +4.9% ✅ index_aspectTime: ✅ 15.248µs (SLO: <20.000µs 📉 -23.8%) vs baseline: -0.1% Memory: ✅ 42.939MB (SLO: <43.250MB 🟡 -0.7%) vs baseline: +4.6% ✅ index_noaspectTime: ✅ 0.462µs (SLO: <10.000µs 📉 -95.4%) vs baseline: -1.0% Memory: ✅ 42.684MB (SLO: <43.000MB 🟡 -0.7%) vs baseline: +4.0% ✅ join_aspectTime: ✅ 16.876µs (SLO: <20.000µs 📉 -15.6%) vs baseline: ~same Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.5% ✅ join_noaspectTime: ✅ 1.501µs (SLO: <10.000µs 📉 -85.0%) vs baseline: +1.4% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.9% ✅ ljust_aspectTime: ✅ 20.734µs (SLO: <30.000µs 📉 -30.9%) vs baseline: -0.2% Memory: ✅ 42.979MB (SLO: <43.250MB 🟡 -0.6%) vs baseline: +4.9% ✅ ljust_noaspectTime: ✅ 2.656µs (SLO: <10.000µs 📉 -73.4%) vs baseline: -0.9% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +5.0% ✅ lower_aspectTime: ✅ 17.757µs (SLO: <30.000µs 📉 -40.8%) vs baseline: -1.3% Memory: ✅ 43.077MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.1% ✅ lower_noaspectTime: ✅ 2.385µs (SLO: <10.000µs 📉 -76.1%) vs baseline: +0.5% Memory: ✅ 42.959MB (SLO: <43.250MB 🟡 -0.7%) vs baseline: +4.7% ✅ lstrip_aspectTime: ✅ 17.542µs (SLO: <20.000µs 📉 -12.3%) vs baseline: -0.4% Memory: ✅ 43.018MB (SLO: <43.250MB 🟡 -0.5%) vs baseline: +4.7% ✅ lstrip_noaspectTime: ✅ 1.848µs (SLO: <10.000µs 📉 -81.5%) vs baseline: +2.1% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.9% ✅ modulo_aspectTime: ✅ 166.805µs (SLO: <200.000µs 📉 -16.6%) vs baseline: +0.3% Memory: ✅ 43.077MB (SLO: <43.500MB 🟡 -1.0%) vs baseline: +5.0% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 174.701µs (SLO: <200.000µs 📉 -12.6%) vs baseline: +0.3% Memory: ✅ 43.096MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +4.7% ✅ modulo_aspect_for_bytesTime: ✅ 167.903µs (SLO: <200.000µs 📉 -16.0%) vs baseline: -0.2% Memory: ✅ 43.136MB (SLO: <43.500MB 🟡 -0.8%) vs baseline: +5.0% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 172.154µs (SLO: <200.000µs 📉 -13.9%) vs baseline: +0.4% Memory: ✅ 43.116MB (SLO: <43.500MB 🟡 -0.9%) vs baseline: +5.1% ✅ modulo_noaspectTime: ✅ 3.691µs (SLO: <10.000µs 📉 -63.1%) vs baseline: ~same Memory: ✅ 43.037MB (SLO: <43.000MB ~same) vs baseline: +4.9% ✅ replace_aspectTime: ✅ 214.776µs (SLO: <300.000µs 📉 -28.4%) vs baseline: ~same Memory: ✅ 43.037MB (SLO: <44.000MB -2.2%) vs baseline: +4.7% ✅ replace_noaspectTime: ✅ 5.765µs (SLO: <10.000µs 📉 -42.3%) vs baseline: 📈 +10.6% Memory: ✅ 42.979MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.8% ✅ repr_aspectTime: ✅ 1.375µs (SLO: <10.000µs 📉 -86.2%) vs baseline: -1.0% Memory: ✅ 42.920MB (SLO: <43.500MB 🟡 -1.3%) vs baseline: +4.7% ✅ repr_noaspectTime: ✅ 0.526µs (SLO: <10.000µs 📉 -94.7%) vs baseline: +0.9% Memory: ✅ 42.939MB (SLO: <43.000MB 🟡 -0.1%) vs baseline: +4.8% ✅ rstrip_aspectTime: ✅ 22.939µs (SLO: <30.000µs 📉 -23.5%) vs baseline: 📈 +20.8% Memory: ✅ 43.057MB (SLO: <43.000MB +0.1%) vs baseline: +5.1% ✅ rstrip_noaspectTime: ✅ 1.884µs (SLO: <10.000µs 📉 -81.2%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.7% ✅ slice_aspectTime: ✅ 16.043µs (SLO: <20.000µs 📉 -19.8%) vs baseline: +0.3% Memory: ✅ 43.018MB (SLO: <43.000MB ~same) vs baseline: +5.0% ✅ slice_noaspectTime: ✅ 0.599µs (SLO: <10.000µs 📉 -94.0%) vs baseline: +0.2% Memory: ✅ 42.782MB (SLO: <43.000MB 🟡 -0.5%) vs baseline: +4.3% ✅ stringio_aspectTime: ✅ 53.865µs (SLO: <80.000µs 📉 -32.7%) vs baseline: +0.1% Memory: ✅ 42.959MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +4.8% ✅ stringio_noaspectTime: ✅ 3.538µs (SLO: <10.000µs 📉 -64.6%) vs baseline: -0.3% Memory: ✅ 42.998MB (SLO: <43.500MB 🟡 -1.2%) vs baseline: +5.0% ✅ strip_aspectTime: ✅ 17.606µs (SLO: <20.000µs 📉 -12.0%) vs baseline: -0.2% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.8% ✅ strip_noaspectTime: ✅ 1.845µs (SLO: <10.000µs 📉 -81.6%) vs baseline: +1.1% Memory: ✅ 43.037MB (SLO: <43.000MB ~same) vs baseline: +4.8% ✅ swapcase_aspectTime: ✅ 18.397µs (SLO: <30.000µs 📉 -38.7%) vs baseline: +0.5% Memory: ✅ 43.018MB (SLO: <43.000MB ~same) vs baseline: +4.7% ✅ swapcase_noaspectTime: ✅ 2.762µs (SLO: <10.000µs 📉 -72.4%) vs baseline: +0.8% Memory: ✅ 43.057MB (SLO: <43.000MB +0.1%) vs baseline: +4.7% ✅ title_aspectTime: ✅ 18.133µs (SLO: <20.000µs -9.3%) vs baseline: -0.4% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +4.8% ✅ title_noaspectTime: ✅ 2.636µs (SLO: <10.000µs 📉 -73.6%) vs baseline: -0.9% Memory: ✅ 42.841MB (SLO: <43.000MB 🟡 -0.4%) vs baseline: +4.4% ✅ translate_aspectTime: ✅ 20.397µs (SLO: <30.000µs 📉 -32.0%) vs baseline: +0.5% Memory: ✅ 42.998MB (SLO: <43.000MB 🟡 ~same) vs baseline: +5.0% ✅ translate_noaspectTime: ✅ 4.240µs (SLO: <10.000µs 📉 -57.6%) vs baseline: +0.4% Memory: ✅ 42.880MB (SLO: <43.000MB 🟡 -0.3%) vs baseline: +4.6% ✅ upper_aspectTime: ✅ 17.734µs (SLO: <30.000µs 📉 -40.9%) vs baseline: -0.7% Memory: ✅ 42.880MB (SLO: <43.000MB 🟡 -0.3%) vs baseline: +4.7% ✅ upper_noaspectTime: ✅ 2.375µs (SLO: <10.000µs 📉 -76.3%) vs baseline: -0.3% Memory: ✅ 43.037MB (SLO: <43.000MB ~same) vs baseline: +5.0% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 5.157µs (SLO: <10.000µs 📉 -48.4%) vs baseline: 📈 +21.9% Memory: ✅ 41.445MB (SLO: <43.000MB -3.6%) vs baseline: +4.9% ✅ ospathbasename_noaspectTime: ✅ 4.331µs (SLO: <10.000µs 📉 -56.7%) vs baseline: +0.8% Memory: ✅ 41.347MB (SLO: <43.000MB -3.8%) vs baseline: +4.5% ✅ ospathjoin_aspectTime: ✅ 6.118µs (SLO: <10.000µs 📉 -38.8%) vs baseline: -0.6% Memory: ✅ 41.543MB (SLO: <43.000MB -3.4%) vs baseline: +5.1% ✅ ospathjoin_noaspectTime: ✅ 6.219µs (SLO: <10.000µs 📉 -37.8%) vs baseline: -0.6% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.9% ✅ ospathnormcase_aspectTime: ✅ 3.509µs (SLO: <10.000µs 📉 -64.9%) vs baseline: -1.4% Memory: ✅ 41.425MB (SLO: <43.000MB -3.7%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 3.583µs (SLO: <10.000µs 📉 -64.2%) vs baseline: +0.6% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.9% ✅ ospathsplit_aspectTime: ✅ 4.836µs (SLO: <10.000µs 📉 -51.6%) vs baseline: +0.2% Memory: ✅ 41.484MB (SLO: <43.000MB -3.5%) vs baseline: +4.9% ✅ ospathsplit_noaspectTime: ✅ 4.938µs (SLO: <10.000µs 📉 -50.6%) vs baseline: -0.3% Memory: ✅ 41.386MB (SLO: <43.000MB -3.8%) vs baseline: +4.8% ✅ ospathsplitdrive_aspectTime: ✅ 3.802µs (SLO: <10.000µs 📉 -62.0%) vs baseline: +0.8% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.9% ✅ ospathsplitdrive_noaspectTime: ✅ 0.749µs (SLO: <10.000µs 📉 -92.5%) vs baseline: +0.7% Memory: ✅ 41.484MB (SLO: <43.000MB -3.5%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 4.646µs (SLO: <10.000µs 📉 -53.5%) vs baseline: +0.2% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +4.9% ✅ ospathsplitext_noaspectTime: ✅ 4.698µs (SLO: <10.000µs 📉 -53.0%) vs baseline: -0.2% Memory: ✅ 41.406MB (SLO: <43.000MB -3.7%) vs baseline: +5.0% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.348µs (SLO: <20.000µs 📉 -83.3%) vs baseline: 📈 +12.1% Memory: ✅ 34.760MB (SLO: <35.500MB -2.1%) vs baseline: +4.7% ✅ 1-count-metrics-100-timesTime: ✅ 201.275µs (SLO: <220.000µs -8.5%) vs baseline: +0.5% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 1-distribution-metric-1-timesTime: ✅ 3.339µs (SLO: <20.000µs 📉 -83.3%) vs baseline: -0.4% Memory: ✅ 34.741MB (SLO: <35.500MB -2.1%) vs baseline: +4.6% ✅ 1-distribution-metrics-100-timesTime: ✅ 216.765µs (SLO: <230.000µs -5.8%) vs baseline: -0.3% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +5.2% ✅ 1-gauge-metric-1-timesTime: ✅ 2.162µs (SLO: <20.000µs 📉 -89.2%) vs baseline: -0.3% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.9% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.133µs (SLO: <150.000µs -9.2%) vs baseline: +0.1% Memory: ✅ 34.859MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.8% ✅ 1-rate-metric-1-timesTime: ✅ 3.163µs (SLO: <20.000µs 📉 -84.2%) vs baseline: +1.1% Memory: ✅ 34.800MB (SLO: <35.500MB 🟡 -2.0%) vs baseline: +4.8% ✅ 1-rate-metrics-100-timesTime: ✅ 215.161µs (SLO: <250.000µs 📉 -13.9%) vs baseline: +0.3% Memory: ✅ 34.898MB (SLO: <35.500MB 🟡 -1.7%) vs baseline: +5.0% ✅ 100-count-metrics-100-timesTime: ✅ 20.067ms (SLO: <22.000ms -8.8%) vs baseline: -1.5% Memory: ✅ 34.996MB (SLO: <35.500MB 🟡 -1.4%) vs baseline: +4.9% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.294ms (SLO: <2.550ms 📉 -10.0%) vs baseline: -0.4% Memory: ✅ 34.839MB (SLO: <35.500MB 🟡 -1.9%) vs baseline: +4.7% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.403ms (SLO: <1.550ms -9.5%) vs baseline: ~same Memory: ✅ 34.957MB (SLO: <35.500MB 🟡 -1.5%) vs baseline: +5.0% ✅ 100-rate-metrics-100-timesTime: ✅ 2.212ms (SLO: <2.550ms 📉 -13.3%) vs baseline: -0.9% Memory: ✅ 34.878MB (SLO: <35.500MB 🟡 -1.8%) vs baseline: +4.9% ✅ flush-1-metricTime: ✅ 4.631µs (SLO: <20.000µs 📉 -76.8%) vs baseline: -0.6% Memory: ✅ 35.134MB (SLO: <35.500MB 🟡 -1.0%) vs baseline: +4.9% ✅ flush-100-metricsTime: ✅ 174.049µs (SLO: <250.000µs 📉 -30.4%) vs baseline: -0.5% Memory: ✅ 35.212MB (SLO: <35.500MB 🟡 -0.8%) vs baseline: +4.9% ✅ flush-1000-metricsTime: ✅ 2.176ms (SLO: <2.500ms 📉 -13.0%) vs baseline: +0.2% Memory: ✅ 35.960MB (SLO: <36.500MB 🟡 -1.5%) vs baseline: +4.9% 🟡 Near SLO Breach (15 suites)🟡 coreapiscenario - 10/10 (1 unstable)
|
Backport fbc2b78 from #15697 to 4.1.
Description
In Python 3.9,
dis.findlinestarts()(used to determine where to insert instrumentation calls) can return imprecise line number values in some corner cases. This PR ensures that we do not explode in such cases (even though the line number information may not be 100% precise).Testing
Unit tests.
Risks
None.
Additional Notes
Perhaps a better solution would be not to rely on
dis.findlinestarts()and implement our own routine for identifying line starts in Python 3.9. But I don't think it's worth doing given that Python 3.9 is itself past its end of life.