Hi.
First off, thank you so much for creating this. It's super helpful for a library I'm building.
After starting to use this, I noticed that my logs for passing tests contained the stdout of markdown snippets in between the tests, so it seems that the plugin is not properly capturing stdout and thus leaking to terminal.
Reproduction
Create test.md
Hello World!
```python
print("SHOULD NOT LEAK")
```
Run it:
➜ pytest-markdown-docs git:(main) ✗ uvx --with pytest-markdown-docs pytest test.md --markdown-docs
======================= test session starts ========================
platform linux -- Python 3.12.7, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/karllorey/git/lorey/pytest-markdown-docs
configfile: pyproject.toml
plugins: markdown-docs-0.9.0
collected 1 item
test.md SHOULD NOT LEAK
. [100%]
======================== 1 passed in 0.02s =========================
expected:
➜ pytest-markdown-docs git:(main) ✗ uvx --with pytest-markdown-docs pytest test.md --markdown-docs
======================= test session starts ========================
platform linux -- Python 3.12.7, pytest-9.0.2, pluggy-1.6.0
rootdir: /home/karllorey/git/lorey/pytest-markdown-docs
configfile: pyproject.toml
plugins: markdown-docs-0.9.0
collected 1 item
test.md . [100%]
======================== 1 passed in 0.02s =========================
Underlying issue
After exploring this with the help of Claude as I'm no expert in pytest, I narrowed this down to how the tests are being run, in particular this code:
# this ensures that pytest's stdout/stderr capture works during the test:
capman = self.config.pluginmanager.getplugin("capturemanager")
with capman.global_and_fixture_disabled():
self.runner.runtest(self.test_definition, all_globals)
from here: https://github.com/modal-labs/pytest-markdown-docs/blob/v0.9.0/src/pytest_markdown_docs/plugin.py#L105
introduced just before the 0.9 release in #43
My understanding is capman.global_and_fixture_disabled() disables capturing stdout and thus results in the leakage As per the pytest docstring:
Context manager to temporarily disable global and current fixture capturing.
This also breaks the capsys fixture in markdown code blocks, since this disables fixture-level capture, too.
Will attempt to create a PR
Hi.
First off, thank you so much for creating this. It's super helpful for a library I'm building.
After starting to use this, I noticed that my logs for passing tests contained the stdout of markdown snippets in between the tests, so it seems that the plugin is not properly capturing stdout and thus leaking to terminal.
Reproduction
Create test.md
Run it:
expected:
Underlying issue
After exploring this with the help of Claude as I'm no expert in pytest, I narrowed this down to how the tests are being run, in particular this code:
from here: https://github.com/modal-labs/pytest-markdown-docs/blob/v0.9.0/src/pytest_markdown_docs/plugin.py#L105
introduced just before the 0.9 release in #43
My understanding is
capman.global_and_fixture_disabled()disables capturing stdout and thus results in the leakage As per the pytest docstring:This also breaks the capsys fixture in markdown code blocks, since this disables fixture-level capture, too.
Will attempt to create a PR