Compatibility: establish baseline and track third_party/tests pass rate

[unclesp1d3r]

Summary

Establish a compatibility test baseline using the GNU file test corpus in `third_party/tests/` and track progress toward the 95% pass rate target.

Context

The test corpus contains 81 test cases (testfile + expected result pairs). Current pass rate with `--use-builtin` is 0% (all tests fail). The compatibility test exists but is `#[ignore]`d and requires the system magic database, not just built-in rules.

Current Results (built-in rules only)

• 0 passed, 81 failed
• 76 tests return "data" (no matching rule)
• 5 tests detect the container format but miss subtypes (ZIP detected but not DOCX/XLSX)

Test Categories Failing

Category	Count	Example
Binary formats (no rule)	~45	RPM, Zstandard, PGP, UF2, xclbin
Text formats (no rule)	~15	JSON, PNM, shell scripts, GEDCOM
Audio formats	~5	MP3, DSD, AMR
ZIP subtypes	~5	DOCX, XLSX, HWP, Keyman
Custom magic tests	~4	regex-eol, searchbug, multiple
Filesystem images	~3	ext4, bcachefs

Acceptance Criteria

• Un-ignore the compatibility test and make it runnable in CI
• Test runs against system text magic files (not just built-in rules)
• CI tracks pass/fail count per run
• Baseline documented (current: 0/81)
• Target: 77/81 (95%) by v1.0

Blocked By

This issue will show progress as the following are implemented:

• Parser: implement comparison operators (<, >, <=, >=) #34 Comparison operators
• Parser: implement bitwise XOR (^), NOT (~), and any-value (x) operators #35 Bitwise operators
• Parser: implement quad (64-bit integer) type with endian variants #36 Quad type
• Evaluator: implement indirect offset resolution #37 Indirect offsets
• Evaluator: implement relative offset resolution #38 Relative offsets
• Parser: implement regex and search types #39 Regex/search types
• Parser: implement date and timestamp types #41 Date types
• Parser: implement default, clear, name, use, and indirect meta-types #42 Meta-types (default, name, use, indirect)

References

• third_party/tests/README for test format documentation
• Epic Epic: libmagic-rs Phase 1 MVP #24

[traycerai]

Warning

Rate limit exceeded. Please try after 2488 seconds.