Skip to content

bug: mutation testing workflow fails with mutmut 3.x incompatibilities #330

@endavis

Description

@endavis

Bug Description

The weekly mutation testing workflow (mutation.yml) fails due to three mutmut 3.x incompatibilities:

  1. Config type mismatch: paths_to_mutate = "src/" (string) in pyproject.toml is iterated as individual characters by mutmut 3.x, producing ['s', 'r', 'c', '/']. The '/' becomes PosixPath('/'), causing mutmut to walk the entire filesystem and crash on dangling symlinks or permission errors.

  2. mutmut html removed: The workflow and task_mutate_html doit task call mutmut html which no longer exists in 3.x.

  3. clear=True env patching breaks mutmut: Tests using patch.dict("os.environ", {}, clear=True) strip the MUTANT_UNDER_TEST env var that mutmut 3.x requires, causing mutmut's initial test run to fail with KeyError: 'MUTANT_UNDER_TEST' and aborting before any mutants are tested.

Steps to Reproduce

  1. Set paths_to_mutate = "src/" (string) in [tool.mutmut] section of pyproject.toml
  2. Have a test that uses patch.dict("os.environ", {}, clear=True)
  3. Run mutmut run
  4. Mutmut crashes walking /, or if that's fixed, aborts because initial test run fails

Expected vs Actual Behavior

Expected: Mutation testing scopes to src/ directory, initial test run passes, mutants are tested.

Actual: Multiple failures:

  • Mutmut walks entire filesystem due to string iteration of paths_to_mutate
  • Tests that clear the environment break mutmut's internal MUTANT_UNDER_TEST check
  • All mutants report "not checked"

Environment

  • mutmut: 3.5.0
  • Runner: ubuntu-latest (Ubuntu 24.04)
  • Python: 3.14

Error Output

Config issue:

FileNotFoundError: [Errno 2] No such file or directory: '/etc/systemd/system/multi-user.target.wants/apport.service'

Environment clearing issue:

FAILED tests/unit/cli/test_completion.py::TestDetectShell::test_detect_no_shell_env - KeyError: 'MUTANT_UNDER_TEST'

Additional Context

  • Root cause for config: paths_to_mutate = "src/" must be paths_to_mutate = ["src/"] (list). Same for tests_dir.
  • Root cause for env: Tests using patch.dict("os.environ", {}, clear=True) must preserve MUTANT_UNDER_TEST when it exists, e.g. patch.dict("os.environ", {k: v for k, v in os.environ.items() if k == "MUTANT_UNDER_TEST"}, clear=True)
  • mutmut html removed in 3.x; task_mutate_html doit task must be removed
  • Workflow needs mkdir -p tmp and artifact path updated from HTML dir to text file
  • Fix validated in bug: mutation testing workflow fails with mutmut 3.x incompatibilities infrafoundry#485

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingneeds-triageNeeds review and prioritization

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions