Byte order mark, U+FEFF

A character that once marked the byte order of a UTF-16 file and now mostly appears at the start of UTF-8 files, where it serves no purpose. Inside a document it is a zero-width no-break space with no visible form.

How it works

ef bb bf in UTF-8. In UTF-16 the code point exists to tell a reader which byte order the file uses; in UTF-8 there is no byte order to signal, so it carries no information and functions purely as a marker that some tools write and others do not expect. Mid-document it behaves as a zero-width no-break space. At the start of a file it is more destructive, because parsers assume the first byte is the first byte: a JSON parser reads it where it expects an opening brace and reports a syntax error pointing at a character you cannot see, a CSV reader includes it in the first column's header name so that lookups by that name fail, and a shell reads it before the shebang so that the script is no longer recognised as executable by that interpreter.

What it is legitimately for

Signalling encoding at the start of a file. Some Windows tools still write it, and some readers still expect it.

What it breaks

• A JSON file beginning with one fails to parse, with an error pointing at the first character of a document that looks fine.
• A CSV's first column header does not match the name you are looking for.
• A shell script beginning with one fails to run because the shebang is no longer the first thing on the line.

How to find it yourself

Look for ef bb bf, especially at the very start of a file.

What Clipboard Sanitizer does

Removed everywhere.