Skip to content

chore: Use json-repair for json_tools tool-call parsing#1071

Open
isaac-cf-wong wants to merge 3 commits intoml-explore:mainfrom
isaac-cf-wong:json_repair
Open

chore: Use json-repair for json_tools tool-call parsing#1071
isaac-cf-wong wants to merge 3 commits intoml-explore:mainfrom
isaac-cf-wong:json_repair

Conversation

@isaac-cf-wong
Copy link
Copy Markdown

Summary

The json_tools parser now uses json_repair.loads() instead of strict JSON parsing so common LLM mistakes (trailing commas, single-quoted strings, etc.) still produce usable tool calls.

Motivation

Chat templates that emit a JSON object inside <tool_call> often get invalid JSON from the model (I encountered this issue frequently when using mlx-community/Qwen3.5-27B-Claude-4.6-Opus-Distilled-MLX-4bit). json.loads fails on those outputs; json-repair is aimed at repairing that kind of text.

Changes

  • mlx_lm/tool_parsers/json_tools.py: parse_tool_call parses the stripped payload with json_repair.loads.
  • setup.py: Declare dependency json-repair>=0.58.7 (latest version).
  • tests/test_tool_parsing.py: Add test_json_tools_repairs_malformed_json covering trailing commas and single-quoted JSON for the multiply and temperature fixtures. Existing test_parsers cases still assert valid JSON for json_tools.

Tradeoffs

Parsing is more lenient than before: some malformed text may be repaired into a JSON object instead of raising, which could occasionally yield unexpected name / arguments. That is usually acceptable for tool-call extraction but is a behavioral change from strict json.loads.

Testing

python -m unittest tests.test_tool_parsing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant