Skip to content

Commit 7fe568f

Browse files
EliahKaganclaude
andcommitted
Add jobs to reproduce Cygwin safe.directory submodule issue
Remove the Cygwin xfail decorations from test_submodules in test_docs.py and test_repo.py, and from test_root_module in test_submodule.py, so the tests surface the underlying failure directly. Add 256 reproduce-safe-dir matrix jobs to cygwin-test.yml, each running these three tests under the current safe.directory configuration. All or most of these reproduce-safe-dir jobs must be removed before this work is integrated. The existing test job's env, defaults, and setup steps gain YAML anchors so the new job can reference them without duplication. Root cause ---------- The CI workflow adds the main repo and its .git directory to safe.directory but not the gitdb or smmap submodule working trees. Git matches safe.directory by exact path. When GitPython opens a submodule as a Repo and runs "git cat-file --batch-check" against it, git rejects the repository for dubious ownership and exits. The race -------- GitPython caches the cat-file process. When git has exited, the next call to __get_object_header (git/cmd.py:1697) hits one of two outcomes depending on whether Python's cmd.stdin.flush() runs before or after the kernel marks the read end of the pipe as closed. (Same mechanism as the long-standing gitpython-developers#427, in which SIGINT kills the cat-file process and the next flush() raises BrokenPipeError.) If Python wins the race, flush() succeeds (data goes into the kernel buffer); the subsequent stdout.readline() returns b"" (EOF); _parse_object_header at git/cmd.py:1659 raises a ValueError whose message names the rejected directory. If git wins, flush() raises BrokenPipeError, which is OSError, which is IOError. In both paths, the exception travels from Object.new_from_sha at git/repo/fun.py:229 (outside name_to_object's try/except for ValueError-from-dereference_recursive) up through repo.commit("HEAD") into iter_items at git/objects/submodule/base.py. That function's "except (IOError, BadName)" at base.py:1597 catches BrokenPipeError but not ValueError. So Path 1 propagates ValueError all the way to the test; Path 2 ends with iter_items returning early and the test seeing an empty submodule list. Per-test outcomes ----------------- What the test sees in Path 2 is determined by what the test does with the empty list: - test_docs::test_submodules does sm.children()[0].name, so the [0] on the empty list raises IndexError at git/util.py:1212. - test_repo::test_submodules does assertGreaterEqual(len(list(self.rorepo.iter_submodules())), 2). The recursive traversal yields gitdb (its iter_items on the *main* repo succeeds, because the main repo IS in safe.directory) but not smmap, so length 1 fails the assertion at test_repo.py:882. - test_submodule::test_root_module does "assert len(rsmsp) >= 2" on a similar traversal result, failing at test_submodule.py:513. The race itself is non-deterministic, but the mapping from race outcome to exception type is deterministic per test, so each test has exactly two possible failure types -- one per side of the race: test_docs: ValueError (Python wins) or IndexError (git wins). test_repo: ValueError (Python wins) or AssertionError (git wins). test_submodule: ValueError (Python wins) or AssertionError (git wins). In particular, test_docs never produces AssertionError, and test_repo and test_submodule never produce IndexError. Empirical confirmation ---------------------- Across 1057 reproduce-safe-dir jobs and 5 buggy-config test (fast) jobs in the runs cited below, every one of 2403 test failures traces to one of four source lines: git/cmd.py:1659 (ValueError, ~98.7%), git/util.py:1212 (IndexError, only test_docs), test/test_repo.py:882 (AssertionError, only test_repo), or test/test_submodule.py:513 (AssertionError, only test_submodule). Zero violations of the per-test prediction. Reproduce-safe-dir runs on the fork: https://github.com/EliahKagan/GitPython/actions/runs/25454533092 https://github.com/EliahKagan/GitPython/actions/runs/25454836713 https://github.com/EliahKagan/GitPython/actions/runs/25472029324 https://github.com/EliahKagan/GitPython/actions/runs/25473762375/attempts/1 https://github.com/EliahKagan/GitPython/actions/runs/25473762375/attempts/2 The first run with the fix applied (256 jobs, 768 test outcomes, all PASSED): https://github.com/EliahKagan/GitPython/actions/runs/25473807645 The independently observed race in the field that prompted this investigation, from a test (fast) job in a PR gitpython-developers#2143 CI run: https://github.com/gitpython-developers/GitPython/actions/runs/25440735020/attempts/1 Plan ---- 1. (this commit) Reproduce the failure under the current safe.directory configuration, with xfails removed so failures surface directly. 2. Apply the fix: extend the safe.directory configuration to cover the gitdb submodule's working tree (and smmap's, recursively). 3. Remove the reproduce-safe-dir jobs and the YAML anchors that only the reproduce job needed. The existing test job retains the permanent fix. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent a714ed6 commit 7fe568f

4 files changed

Lines changed: 53 additions & 30 deletions

File tree

.github/workflows/cygwin-test.yml

Lines changed: 53 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -20,64 +20,74 @@ jobs:
2020

2121
runs-on: windows-latest
2222

23-
env:
23+
env: &cygwin-env
2424
CHERE_INVOKING: "1"
2525
CYGWIN_NOWINPATH: "1"
2626

27-
defaults:
27+
defaults: &cygwin-defaults
2828
run:
2929
shell: C:\cygwin\bin\bash.exe --login --norc -eo pipefail -o igncr "{0}"
3030

3131
steps:
32-
- name: Force LF line endings
32+
- &force-lf
33+
name: Force LF line endings
3334
run: |
3435
git config --global core.autocrlf false # Affects the non-Cygwin git.
3536
shell: pwsh # Do this outside Cygwin, to affect actions/checkout.
3637

37-
- uses: actions/checkout@v6
38+
- &checkout
39+
uses: actions/checkout@v6
3840
with:
3941
fetch-depth: 0
4042
submodules: recursive
4143

42-
- name: Install Cygwin
44+
- &install-cygwin
45+
name: Install Cygwin
4346
uses: cygwin/cygwin-install-action@v6
4447
with:
4548
packages: git python39 python-pip-wheel python-setuptools-wheel python-wheel-wheel
4649
add-to-path: false # No need to change $PATH outside the Cygwin environment.
4750

48-
- name: Arrange for verbose output
51+
- &verbose-output
52+
name: Arrange for verbose output
4953
run: |
5054
# Arrange for verbose output but without shell environment setup details.
5155
echo 'set -x' >~/.bash_profile
5256
53-
- name: Special configuration for Cygwin git
57+
- &safe-directory
58+
name: Special configuration for Cygwin git
5459
run: |
5560
git config --global --add safe.directory "$(pwd)"
5661
git config --global --add safe.directory "$(pwd)/.git"
5762
git config --global core.autocrlf false
5863
59-
- name: Prepare this repo for tests
64+
- &prepare-repo
65+
name: Prepare this repo for tests
6066
run: |
6167
./init-tests-after-clone.sh
6268
63-
- name: Set git user identity and command aliases for the tests
69+
- &git-identity
70+
name: Set git user identity and command aliases for the tests
6471
run: |
6572
git config --global user.email "travis@ci.com"
6673
git config --global user.name "Travis Runner"
6774
# If we rewrite the user's config by accident, we will mess it up
6875
# and cause subsequent tests to fail
6976
cat test/fixtures/.gitconfig >> ~/.gitconfig
7077
71-
- name: Set up virtual environment
78+
- &setup-venv
79+
name: Set up virtual environment
7280
run: |
7381
python3.9 -m venv .venv
7482
echo 'BASH_ENV=.venv/bin/activate' >>"$GITHUB_ENV"
7583
76-
- name: Update PyPA packages
84+
- &update-pypa
85+
name: Update PyPA packages
7786
run: |
7887
python -m pip install -U pip 'setuptools; python_version<"3.12"' wheel
7988
80-
- name: Install project and test dependencies
89+
- &install-deps
90+
name: Install project and test dependencies
8191
run: |
8292
pip install '.[test]'
8393
@@ -92,3 +102,34 @@ jobs:
92102
- name: Test with pytest (${{ matrix.additional-pytest-args }})
93103
run: |
94104
pytest --color=yes -p no:sugar --instafail -vv ${{ matrix.additional-pytest-args }}
105+
106+
reproduce-safe-dir:
107+
strategy:
108+
matrix:
109+
run: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256]
110+
fail-fast: false
111+
112+
runs-on: windows-latest
113+
114+
env: *cygwin-env
115+
116+
defaults: *cygwin-defaults
117+
118+
steps:
119+
- *force-lf
120+
- *checkout
121+
- *install-cygwin
122+
- *verbose-output
123+
- *safe-directory
124+
- *prepare-repo
125+
- *git-identity
126+
- *setup-venv
127+
- *update-pypa
128+
- *install-deps
129+
130+
- name: Run submodule tests
131+
run: |
132+
python -m pytest -vv \
133+
test/test_docs.py::Tutorials::test_submodules \
134+
test/test_repo.py::TestRepo::test_submodules \
135+
test/test_submodule.py::TestSubmodule::test_root_module

test/test_docs.py

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,6 @@
66
import gc
77
import os
88
import os.path
9-
import sys
10-
11-
import pytest
129

1310
from test.lib import TestBase
1411
from test.lib.helper import with_rw_directory
@@ -478,11 +475,6 @@ def test_references_and_objects(self, rw_dir):
478475

479476
repo.git.clear_cache()
480477

481-
@pytest.mark.xfail(
482-
sys.platform == "cygwin",
483-
reason="Cygwin GitPython can't find SHA for submodule",
484-
raises=ValueError,
485-
)
486478
def test_submodules(self):
487479
# [1-test_submodules]
488480
repo = self.rorepo

test/test_repo.py

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -877,11 +877,6 @@ def test_repo_odbtype(self):
877877
target_type = GitCmdObjectDB
878878
self.assertIsInstance(self.rorepo.odb, target_type)
879879

880-
@pytest.mark.xfail(
881-
sys.platform == "cygwin",
882-
reason="Cygwin GitPython can't find submodule SHA",
883-
raises=ValueError,
884-
)
885880
def test_submodules(self):
886881
self.assertEqual(len(self.rorepo.submodules), 1) # non-recursive
887882
self.assertGreaterEqual(len(list(self.rorepo.iter_submodules())), 2)

test/test_submodule.py

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -480,11 +480,6 @@ def test_base_rw(self, rwrepo):
480480
def test_base_bare(self, rwrepo):
481481
self._do_base_tests(rwrepo)
482482

483-
@pytest.mark.xfail(
484-
sys.platform == "cygwin",
485-
reason="Cygwin GitPython can't find submodule SHA",
486-
raises=ValueError,
487-
)
488483
@pytest.mark.xfail(
489484
HIDE_WINDOWS_KNOWN_ERRORS,
490485
reason=(

0 commit comments

Comments
 (0)