Fix file -m/-M options: support hex escapes and correct magic file selection

[Copilot]

The -m and -M options were non-functional. When -m was specified, both the custom and default magic files were used instead of just the custom file. Additionally, hex escape sequences (\xNN) in magic files were not parsed, preventing detection of binary formats like JBIG2.

Changes

Magic file selection logic

• -m file now uses only the specified file (previously added default as well)
• -m file -d uses both the specified file and default (correct behavior)
• -M behaves identically to -m per POSIX spec

Hex escape sequence support

• Implemented \xNN parsing for magic file patterns
• Example: 0 string \x97JB2\x0D\x0A\x1A\x0A JBIG2 image data now works

Binary data handling

• Changed Value::String from String to Vec<u8> to support arbitrary byte sequences
• Rust's String type requires valid UTF-8, but magic patterns match raw binary data
• Rewrote octal escape parsing to avoid UTF-8 encoding corruption for bytes >0x7F
• String comparison now uses raw bytes instead of UTF-8 conversion

Example

Before:

$ file -m jbig2.magic test.jb2
test.jb2: data

After:

$ file -m jbig2.magic test.jb2
test.jb2: JBIG2 image data

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• pubs.opengroup.org
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

---

This section details on the original issue you should resolve

<issue_title>file utility</issue_title>
<issue_description>-m and -M options don't work

file -m jbig2.magic test.jb2
test:.jb2 : data

Expected output:
file -m jbig2.magic test.jb2
test.jb2: JBIG2 image data</issue_description>

<agent_instructions>Review POSIX spec for file then proceed</agent_instructions>

Comments on the Issue (you are @copilot in this section)

@jgarzik `file` is a challenge, and I am not sure what is the best solution.

Linux and macOS both have different, OS specific binary magic file formats. macOS's is undocumented, I believe.

We could ship our own full magic database, but that adds an unwanted maintenance burden, a database constantly hungry for updates. Other options (parsing OS binary C structures) are ugly. :) </comment_new>

• Fixes file utility #537

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[jgarzik]

this change breaks ci:

test ui::terminal::tests::test_both_env_override ... FAILED

failures:

---- ui::terminal::tests::test_both_env_override stdout ----

thread 'ui::terminal::tests::test_both_env_override' (20992) panicked at editors/vi/ui/terminal.rs:334:13:
assertion `left == right` failed
  left: 80
 right: 100
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
   4: core::ops::function::FnOnce::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    ui::terminal::tests::test_both_env_override

test result: FAILED. 177 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

error: test failed, to rerun pass `-p posixutils-editors --lib`
Error: Process completed with exit code 101.

[Copilot]

this change breaks ci:

...

The CI failure in editors/vi/ui/terminal.rs:334 is unrelated to this PR. My changes only modified files in the file/ directory and did not touch any code in the editors/ module.

I ran the failing test locally and it passes:

test ui::terminal::tests::test_both_env_override ... ok

All 178 tests in posixutils-editors pass, and all 77 tests in posixutils-file pass. This appears to be a flaky test or environment-specific issue in CI, not caused by these changes.