Fix issue #1022: pytest-xdist + --profile-all on Linux dropped user samples (#1055)

execnet worker subprocesses spawned by pytest-xdist inherited a sigmask
with Scalene's CPU sampling signal blocked. The parent's
patch_module_functions_with_signal_blocking shim (added for issue #841)
wraps every os.* call with pthread_sigmask(SIG_BLOCK, [SIGALRM]); when
that wrapper intersects subprocess.Popen's fork window, the child
inherits the blocked mask. setitimer fires every 10 ms, but the kernel
keeps the signal pending and the handler never runs — workers collect
zero CPU and zero allocation samples, each prints "did not run for long
enough", and the user's source file is absent from the merged profile.

Fix in enable_signals(): unconditionally pthread_sigmask(SIG_UNBLOCK, ...)
Scalene's profiling signals in the calling (main) thread so children
clear any inherited block before the timer starts.

Adds a Linux-only smoketest that runs --profile-all with -n 2 xdist
workers against a CPU-busy fixture and asserts the user file ends up
in the profile. Pre-fix the smoketest exits 1 with the exact warning
the bug reporter saw; post-fix it passes.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: emeryberger

.github/workflows/test-smoketests.yml

@@ -68,6 +68,17 @@ jobs:
       run: python test/smoketest_pool_spawn.py
       timeout-minutes: 5
 
+    # Regression test for pytest-xdist + --profile-all on Linux (issue
+    # #1022). The bug was Linux-specific: execnet worker subprocesses
+    # inherited a sigmask with Scalene's CPU sampling signal blocked,
+    # so the timer fired but the handler never ran and the user's code
+    # was absent from the merged profile. The smoketest itself self-
+    # skips on non-Linux platforms.
+    - name: pytest-xdist + --profile-all (issue #1022)
+      if: matrix.os == 'ubuntu-latest'
+      run: python test/smoketest_issue_1022.py
+      timeout-minutes: 5
+
       # Note: test/smoketest.py only handles single JSON, rather than multiple in sequence.
     - name: profile-interval smoke test
       run: python -m scalene run --profile-interval=2 test/testme.py && python -m scalene view --cli

scalene/scalene_signal_manager.py

@@ -238,6 +238,24 @@ def enable_signals(
             self.__signals.cpu_timer_signal,
             cpu_sampling_rate,
         )
+        # Unmask Scalene's profiling signals in the calling (main) thread.
+        # pytest-xdist's execnet, and other libraries that fan out work via
+        # subprocess.Popen, can leave the worker process with Scalene's CPU
+        # sampling signal blocked in its inherited sigmask — most visibly
+        # when the parent's patched os module (issue #841) intersects with
+        # the subprocess fork window. Without this unblock the SIGALRM
+        # timer fires but the handler never runs, leaving the worker with
+        # zero CPU and zero allocation samples. See issue #1022.
+        unblock_set = {
+            self.__signals.cpu_signal,
+            self.__signals.malloc_signal,
+            self.__signals.free_signal,
+            self.__signals.memcpy_signal,
+        }
+        unblock_set.discard(None)
+        if unblock_set:
+            with contextlib.suppress(ValueError, OSError):
+                signal.pthread_sigmask(signal.SIG_UNBLOCK, unblock_set)
 
     def setup_lifecycle_signals(
         self,

test/smoketest_issue_1022.py

@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+"""Smoketest for pytest-xdist + ``--profile-all`` (regression for #1022).
+
+Workers spawned by pytest-xdist inherit a sigmask that, in combination
+with Scalene's ``patch_module_functions_with_signal_blocking`` shim
+(issue #841), used to land with Scalene's CPU sampling signal blocked.
+The setitimer fires, but the handler never runs, so each worker
+collects zero samples; the parent then merges nothing and the user's
+code is absent from the profile while ``"did not run for long enough"``
+warnings print twice (once per worker).
+
+This smoketest reproduces the exact shape: ``-n 2`` workers running a
+CPU-busy fixture under ``--profile-all``. It asserts that the user's
+source file appears in the merged profile. Pre-fix on Linux 3.13 the
+profile contains only stdlib/execnet; post-fix it contains the fixture
+source.
+
+Linux-only: macOS and Windows did not exhibit the bug in the wild
+(macOS goes through a different path that doesn't leak the sigmask
+across subprocess fork; Windows doesn't use the LD_PRELOAD/sigmask
+machinery at all).
+"""
+
+import json
+import os
+import subprocess
+import sys
+import tempfile
+import textwrap
+
+if sys.platform != "linux":
+    print(f"skipping issue-1022 smoketest on {sys.platform}: bug is Linux-only")
+    sys.exit(0)
+
+# Install pytest-xdist into the running environment if missing. The
+# smoketest workflow already installs scalene + numpy; xdist is the
+# only extra requirement and pinning it isn't worth a separate step.
+try:
+    import xdist  # noqa: F401
+except ImportError:
+    subprocess.check_call(
+        [sys.executable, "-m", "pip", "install", "--quiet", "pytest-xdist"]
+    )
+
+workdir = tempfile.mkdtemp(prefix="scalene-1022-")
+src_dir = os.path.join(workdir, "src")
+os.makedirs(src_dir, exist_ok=True)
+
+# Fixture: a CPU-busy function the workers will run. Each test calls it
+# with 10M iterations so even on slow runners there is more than enough
+# wall-clock time for the timer to fire many times per worker.
+with open(os.path.join(src_dir, "__init__.py"), "w") as f:
+    f.write(
+        textwrap.dedent(
+            """
+            def crunch(n):
+                total = 0
+                for i in range(n):
+                    total += i * i
+                return total
+            """
+        ).lstrip()
+    )
+
+with open(os.path.join(workdir, "test_heavy.py"), "w") as f:
+    f.write(
+        textwrap.dedent(
+            """
+            from src import crunch
+
+            def test_a():
+                assert crunch(10_000_000) > 0
+
+            def test_b():
+                assert crunch(10_000_000) > 0
+
+            def test_c():
+                assert crunch(10_000_000) > 0
+
+            def test_d():
+                assert crunch(10_000_000) > 0
+            """
+        ).lstrip()
+    )
+
+profile_path = os.path.join(workdir, "issue1022.json")
+cmd = [
+    sys.executable,
+    "-m",
+    "scalene",
+    "run",
+    "--profile-all",
+    "-o",
+    profile_path,
+    "---",
+    "-m",
+    "pytest",
+    "-n",
+    "2",
+    "test_heavy.py",
+]
+print("COMMAND", " ".join(cmd))
+proc = subprocess.run(cmd, cwd=workdir, capture_output=True, text=True, timeout=180)
+print(proc.stdout)
+print(proc.stderr, file=sys.stderr)
+
+if proc.returncode != 0:
+    print(f"scalene exited with rc={proc.returncode}")
+    sys.exit(proc.returncode)
+
+if not os.path.exists(profile_path):
+    print(f"No profile produced at {profile_path}")
+    sys.exit(1)
+
+with open(profile_path) as f:
+    data = json.load(f)
+
+files = data.get("files", {})
+if not files:
+    print("Profile has empty 'files' dict — issue #1022 regression")
+    sys.exit(1)
+
+# The fixture lives in workdir; the entry we expect to see is
+# "<workdir>/src/__init__.py" since that's where ``crunch`` runs. We
+# match by path suffix to stay tolerant of symlinked tmpdirs.
+user_file_suffix = os.path.join("src", "__init__.py")
+matching = [name for name in files if name.endswith(user_file_suffix)]
+if not matching:
+    print(
+        f"User-code file (...{user_file_suffix}) absent from profile. "
+        f"Files present: {sorted(files)}"
+    )
+    sys.exit(1)
+
+print(
+    f"OK: issue #1022 fix holds. Profiled {len(files)} files, "
+    f"user code at {matching[0]!r}, elapsed={data.get('elapsed_time_sec'):.2f}s"
+)
+sys.exit(0)