Perf regression: SQL COUNT(*) over a CSV scan lost its parallelism in 1.44.x (select(pl.len()) unaffected)

[jqnatividad]

Checks

• I have checked that this issue has not already been reported.
• I have confirmed this bug exists on the latest version of Polars.

Reproducible example

COUNT(*) issued through SQLContext over a CSV scan stopped running in parallel in 1.44.x. The
equivalent scan_csv(...).select(pl.len()) is unaffected, so the fast row-count itself still works —
only the SQL route lost its threading.

Generate a CSV (1M rows x 10 cols, ~105 MB):

import polars as pl
n = 1_000_000
df = pl.DataFrame({f"c{i}": pl.Series([f"value_{i}_{j%997}" for j in range(n)]) for i in range(8)})
df = df.with_columns(pl.int_range(n).alias("id"), (pl.int_range(n) % 28).alias("grp"))
df.write_csv("synth.csv")

Benchmark, reporting CPU time alongside wall time (the cpu/wall ratio is the point):

import time, polars as pl
path = "synth.csv"

def bench(label, fn):
    fn()  # warm
    best = None
    for _ in range(3):
        w0, c0 = time.perf_counter(), time.process_time()
        out = fn()
        w, c = time.perf_counter() - w0, time.process_time() - c0
        if best is None or w < best[0]:
            best = (w, c, out)
    w, c, out = best
    print(f"{label:34s} wall={w*1000:7.1f} ms  cpu={c*1000:7.1f} ms  cpu/wall={c/w:5.2f}  -> {out}")

def sql_registered():
    ctx = pl.SQLContext()
    ctx.register("t", pl.scan_csv(path))
    return ctx.execute("SELECT COUNT(*) FROM t").collect().item()

print(f"=== polars {pl.__version__} ===")
bench("A scan_csv.select(len())",       lambda: pl.scan_csv(path).select(pl.len()).collect().item())
bench("B SQL COUNT(*) read_csv()",      lambda: pl.SQLContext().execute(f"SELECT COUNT(*) FROM read_csv('{path}')").collect().item())
bench("C SQL COUNT(*) registered scan", sql_registered)
bench("D group_by (control)",           lambda: pl.scan_csv(path).group_by("grp").len().collect().height)

Log output

=== polars 1.43.2 ===
A scan_csv.select(len())           wall=    2.3 ms  cpu=   29.8 ms  cpu/wall=12.95  -> 1000000
B SQL COUNT(*) read_csv()          wall=    3.3 ms  cpu=   32.5 ms  cpu/wall= 9.83  -> 1000000
C SQL COUNT(*) registered scan     wall=    2.3 ms  cpu=   31.2 ms  cpu/wall=13.64  -> 1000000
D group_by (control)               wall=   23.3 ms  cpu=  140.5 ms  cpu/wall= 6.02  -> 28

=== polars 1.44.2 ===
A scan_csv.select(len())           wall=    2.2 ms  cpu=   30.8 ms  cpu/wall=13.85  -> 1000000
B SQL COUNT(*) read_csv()          wall=   22.3 ms  cpu=   30.3 ms  cpu/wall= 1.36  -> 1000000   <-- 6.8x slower
C SQL COUNT(*) registered scan     wall=   18.4 ms  cpu=   25.9 ms  cpu/wall= 1.41  -> 1000000   <-- 8x slower
D group_by (control)               wall=   27.0 ms  cpu=  143.7 ms  cpu/wall= 5.32  -> 28

Issue description

Total CPU time is unchanged — only the parallelism is gone. B goes from cpu/wall 9.83 to 1.36 and
C from 13.64 to 1.41, while the CPU column barely moves. Nothing is doing more work; the work simply
stopped being spread across threads.

Scope:

• A (scan_csv(...).select(pl.len())) is not affected — still ~13x parallel on both versions.
• B and C are affected identically, so this is not specific to the read_csv() table
  function; plain SELECT COUNT(*) over a registered scan_csv LazyFrame regresses the same way.
• D, a real aggregation over the same file, is unaffected — so general CSV scanning and
  group-by parallelism are fine.

Version bisect on released wheels, same machine and file:

version	B SQL COUNT(*) read_csv()	cpu/wall
1.43.2	3.3 ms	9.83
1.44.0	(yanked, untested)
1.44.1	21.8 ms	1.33
1.44.2	22.3 ms	1.36
2.0.0rc1	20.2 ms	1.38

All rows above are the same machine and the same synth.csv from the repro.
2.0.0rc1 is still affected, so this is live on main.

Regression window: py-1.43.2 (2026-07-31) → py-1.44.1 (2026-08-25), 132 commits.

It looks like COUNT(*) coming out of the SQL planner no longer lowers to the parallel FastCount
path that pl.len() still reaches. This same class of bug has been fixed twice before — #22363 and
#27504 ("Fix perf regression in scan_csv select(len()) when collected on streaming engine",
reporting the same 92% cpu → 587% cpu signature). 54ecaed (the #27504 fix) is an ancestor of
py-1.44.2, so that fix is present and this is a distinct regression on the SQL side.

Found via qsv, whose un-indexed count uses
SELECT COUNT(*) FROM read_csv(...); it regressed 8.6x on release
(dathere/qsv#4603).

Expected behavior

SELECT COUNT(*) over a CSV scan should lower to the same parallel fast-count path as
select(pl.len()) and stay within noise of it (~2-3 ms on this file), rather than running serially.

Workaround for anyone else hitting this: use scan_csv(...).select(pl.len()) instead of SQL
COUNT(*) — row A shows it is unaffected on every version tested, including 2.0.0rc1.

Installed versions

Measured on macOS 27 (Darwin 27.0.0), Apple M4 Max, 16 cores, warm page cache; wheels installed
via uv run --with polars==<version>.

pl.show_versions()

--------Version info---------
Polars:              1.44.2
Index type:          UInt32
Platform:            macOS-27.0-arm64-arm-64bit
Python:              3.12.10 (v3.12.10:0cc81280367, Apr  8 2025, 08:46:59) [Clang 13.0.0 (clang-1300.0.29.30)]
Runtime:             rt32

----Optional dependencies----
Azure CLI            <not installed>
adbc_driver_manager  <not installed>
altair               <not installed>
azure.identity       <not installed>
boto3                <not installed>
cloudpickle          <not installed>
connectorx           <not installed>
deltalake            <not installed>
fastexcel            <not installed>
fsspec               2026.3.0
gevent               <not installed>
google.auth          <not installed>
great_tables         <not installed>
matplotlib           <not installed>
numpy                2.4.1
openpyxl             <not installed>
pandas               2.3.3
polars_cloud         <not installed>
pyarrow              <not installed>
pydantic             2.13.1
pyiceberg            <not installed>
sqlalchemy           <not installed>

Note

Prepared with Opus 5.1