Skip to content

fix(pytest): surface collection errors instead of 'No tests collected'#2850

Open
mariofeter wants to merge 1 commit into
rtk-ai:developfrom
mariofeter:fix/pytest-collection-errors
Open

fix(pytest): surface collection errors instead of 'No tests collected'#2850
mariofeter wants to merge 1 commit into
rtk-ai:developfrom
mariofeter:fix/pytest-collection-errors

Conversation

@mariofeter

@mariofeter mariofeter commented Jul 6, 2026

Copy link
Copy Markdown

Problem

The pytest summarizer returns Pytest: No tests collected whenever the summary line contains no passed/failed/skipped counts. This hides:

  • Collection errors (ImportError/SyntaxError in a test module): pytest prints an ERRORS section and 1 error in 0.06s, exits 2 — rtk reports "No tests collected" and the root cause is lost. This is exactly the RED case TDD workflows rely on.
  • Non-session output: rtk pytest --version also reports "No tests collected" instead of the version string.

Repro (pytest 9, real output fixture included):

$ rtk pytest tests/        # tests/test_broken.py has a broken import
Pytest: No tests collected   # before — root cause invisible

Fix

  • parse_summary_line now parses N error(s) counts (both === 1 error in 0.12s === and bare -q 1 error in 0.06s; the bare form requires a leading integer so traceback prose can't masquerade as a summary line)
  • the ERRORS section is treated like FAILURES, so collection tracebacks are captured
  • ERROR entries render under an Errors: header with an [ERR] tag
  • root-cause lines (> / E markers) are selected before traceback location boilerplate within the existing 3-line cap, so E ModuleNotFoundError: ... always survives
  • unrecognized non-session output (--version, --help, plugin banners) is passed through unchanged; "No tests collected" now requires pytest to have actually said so (no tests ran / collected 0 items)
  • !!! Interrupted !!! banners are skipped in failure details (the error count already lands in the header)
  • dropped the dead test_files collection (was already unused: _test_files)

After

$ rtk pytest tests/
Pytest: 0 passed, 0 failed, 1 error

Errors:
1. [ERR] collecting tests/test_broken.py
     ImportError while importing test module '/tmp/proj/tests/test_broken.py'.
     /usr/lib/python3.14/importlib/__init__.py:88: in import_module
     E   ModuleNotFoundError: No module named 'nonexistent_module'

$ rtk pytest --version
pytest 9.0.3

Green runs are unchanged (Pytest: N passed), normal assertion failures unchanged.

Test plan

  • New unit tests: quiet-mode collection error (real pytest 9 fixture), banner-mode collection error, --version passthrough, parse_summary_line error counts
  • All existing pytest filter tests pass (24 total)
  • cargo fmt --all && cargo clippy --all-targets clean
  • Manual: collection error (exit 2 propagated), green suite, mixed pass/fail, --version

The pytest summarizer returned 'Pytest: No tests collected' whenever the
summary line had no passed/failed/skipped counts, hiding collection
errors (ImportError/SyntaxError in test modules) and swallowing
non-session output like --version. Exactly the RED case TDD relies on.

- parse 'N error(s)' counts from summary lines (=== wrapped and -q bare)
- treat the ERRORS section like FAILURES so tracebacks are collected
- render ERROR entries with [ERR] tag and an 'Errors:' section header
- prioritize '>'/'E' root-cause lines over traceback boilerplate within
  the 3-line cap
- pass through unrecognized non-session output (--version, --help)
  instead of mislabeling it; 'No tests collected' now requires pytest
  to have actually said so
- skip '!!! Interrupted !!!' banner lines in failure details

Real pytest 9 fixture included (--tb=short -q -rxX collection error).
@CLAassistant

CLAassistant commented Jul 6, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants