UTF-8 and BOM on Mac: Identify Encoding Before Editing a Manuscript

Three same-content manuscripts reveal BOM and line-ending differences through an abstract byte-inspection prism

Before editing a UTF-8 manuscript on a Mac, observe three independent facts: whether the bytes are valid as UTF-8, whether a BOM is present, and whether a representative line ends in LF or CRLF. This article stops at observation. It does not convert, repair, rank editors, or approve a changed file.

Use xxd and iconv without writing to the manuscript. The values below were checked on macOS on August 11, 2026 as general command results; they are not Rune Studio measurements.

Check whether the bytes are valid as UTF-8

For manuscript.txt, run:

iconv -f UTF-8 -t UTF-8 manuscript.txt >/dev/null

An error identifies bytes that are invalid under UTF-8. Success does not prove that UTF-8 was the original encoding, because an ASCII-only file can be valid under several encodings. Record “valid as UTF-8,” then keep the sender’s declaration and the editor’s encoding label as separate evidence.

Read only the first three bytes for a BOM

xxd -l 3 -g 1 manuscript.txt

The UTF-8 BOM is ef bb bf. A match means “UTF-8 BOM present”; a different prefix means “no UTF-8 BOM.” BOM absence alone does not identify the encoding.

For the controlled line 校正開始 in UTF-8/LF, the no-BOM sample began e6 a0 a1, while the BOM sample began ef bb bf e6 a0 a1. These are control values, not values claimed for a reader’s file.

Keep newline bytes separate from the BOM

Inspect a representative line with xxd -g 1 manuscript.txt. LF ends in 0a; CRLF ends in 0d 0a. Record them independently, such as “BOM present / LF” or “no BOM / CRLF.”

The controlled UTF-8/LF line ended ... e5 a7 8b 0a; the CRLF version ended ... e5 a7 8b 0d 0a. A full-file mixed-newline audit belongs to a separate inspection; this article records the observed representative line.

Separate raw values from interpretation

Observation Raw value Interpretation Caution
UTF-8 validity iconv exit result valid / error not proof of origin
First three bytes for example ef bb bf BOM present / absent not a BOM policy decision
Representative ending for example 0d 0a CRLF / LF separate from BOM
Target identity filename and SHA-256 observed revision never reuse for another revision

Write “not observed,” not “absent,” when a value was not collected. shasum -a 256 manuscript.txt gives the identifier used to bind the card to one revision.

Keep the exact command on the card and quote filenames that contain spaces. A screenshot without the target hash cannot identify which revision produced the output. If mixed LF and CRLF are possible, record only the representative line and mark the full-file state unobserved. A prefix other than ef bb bf supports “no BOM”; it does not, by itself, support “encoding proven as UTF-8.”

Rune Studio’s boundary

Stage 3 materials confirm that Rune Studio detects major Japanese encodings, displays unconvertible characters, and stops automatic saving while an issue remains. They do not establish automatic BOM, leading-byte, or LF/CRLF inspection.

Therefore the xxd readings remain general observations, not product results. STUDIO-534 owns conversion, STUDIO-065 owns the save-block workflow, and STUDIO-006 owns accident classification.

Rune Studio editor with UTF-8 and LF shown in the status area
The status area shows UTF-8 and LF for the open manuscript. This is the selected value; whether the saved bytes match is checked separately.

Finish with a pre-change observation card

Observation is complete when target identity, UTF-8 validity, the first three bytes, and the representative line ending contain raw values and clearly mark anything unknown. Adding or removing a BOM, normalizing newlines, or saving another file happens after this handoff, outside this article. Add the observation time and macOS environment so a later reading can explain differences under the same conditions.