The Invisible Characters Breaking Your Text

An invisible character is a real character that renders as blank, or as nothing at all. The usual suspects are U+00A0, the non-breaking space; U+200B, the zero-width space; U+FEFF, the byte order mark; and the eleven typographic spaces between U+2000 and U+200A. Your eyes skip them. Every string comparison a computer makes counts them. That is why a spreadsheet lookup fails on a name you can plainly read, why a pasted password is rejected, and why two lines that look identical refuse to collapse into one.

The symptom always has the same shape: two things that look the same are not the same. Nothing is wrong with the software, or with your eyes. There is an extra character in there and it has no picture.

Which characters are invisible?

Two groups, with different causes.

Spaces that are not the space bar

Unicode's space separator category holds 17 characters. Exactly one of them is the key on your keyboard. The other 16 look like a space, measure like a space, and fail to match a space.

• U+00A0, the non-breaking space. The most common by a wide margin. It arrives from   in HTML, from Word's autoformatting, and on a Mac from Option plus the space bar. In UTF-8 it is 2 bytes, C2 A0, where an ordinary space is one.
• U+202F, the narrow no-break space. French typography puts one before a colon, semicolon, question mark or exclamation mark, so French text copied from almost anywhere is full of them.
• U+2000 to U+200A. Eleven fixed typesetting widths — en space, em space, three-per-em, thin space, hair space, figure space — that come out of layout software and PDFs.
• U+3000, the ideographic space. What the space bar produces in a Japanese or Chinese input method. It is full width, so it looks like two spaces rather than a suspicious one.

Characters with no width at all

These do not even look like a space. They occupy zero pixels, so text containing one is visually identical to text without it.

• U+200B, the zero-width space. A hint that says "you may break the line here". Content management systems insert them inside long words and URLs so they wrap.
• U+200C and U+200D, the zero-width non-joiner and joiner. They control whether neighbouring characters fuse. Unlike the rest of this list they do real work, which matters later.
• U+2060, the word joiner. The opposite of a zero-width space: it forbids a break.
• U+FEFF, the byte order mark. Three bytes, EF BB BF, that belong at the start of a file to announce its encoding. Editors add one whether you asked or not, and it ends up mid-string whenever two files are concatenated.

How do I see invisible characters?

You cannot, directly. You detect them three ways.

Count them. Paste the text into the whitespace cleaner here and it reports what it found before it changes anything: how many non-breaking spaces, how many other Unicode spaces, how many zero-width characters, how many lines end in a space. An empty report rules out the whole category in ten seconds.

Compare the length with what you expect. A product code that should be 8 characters and reports 9 has a passenger.

Turn on your editor's highlighting. VS Code boxes in invisible and ambiguous characters by default, a feature it added after the Trojan Source disclosure in 2021 showed that source code can be made to read one way to a human and another to a compiler. Most editors have a version of this, under "show whitespace" or "render control characters".

Why does find and replace not remove them?

Because the shortcuts you would reach for disagree about what counts as whitespace.

In JavaScript, \s and trim() both match U+00A0 and U+FEFF but not U+200B, which is classified as a format character rather than a space. Python behaves the same way: '\xa0'.isspace() is true, '\u200b'.isspace() is false, so .strip() leaves it behind. Excel is worse. TRIM removes only the ASCII space, character 32, and CLEAN only characters 0 to 31, so neither touches the non-breaking space at 160. There you need SUBSTITUTE(A1,CHAR(160)," ") before TRIM does anything useful.

To target them explicitly, use a regex. Tick Regular expression in the find and replace tool and search for [\u00A0\u200B\u200C\u200D\u2060\uFEFF] — the zero-width set plus the non-breaking space. Read the match count before you replace anything.

Where do they come from?

Almost always a copy and paste across a boundary: a web page, where every   copies out as a real U+00A0; a Word document, where autoformatting adds them around numbers and initials; a PDF, which carries whatever spacing the typesetter used.

The byte order mark is the exception, and it breaks parsers rather than comparisons. Excel's UTF-8 CSV export puts one at the front of the file. JSON.parse throws on a leading U+FEFF, since JSON allows only space, tab, carriage return and line feed as whitespace. Python's csv module hands you a first field called \ufeffid unless you open the file with the utf-8-sig encoding, which exists for exactly this. If a CSV conversion produces one key with a name you cannot see anything wrong with, check that first.

How do I strip them out?

For prose, replace every exotic space with an ordinary one and delete the zero-width characters outright. That is the default in the cleaner: "Replace exotic spaces" covers all 16 non-standard space separators, "Remove invisible characters" covers the zero-width set and the byte order mark, and the before and after counts tell you how many left.

For code, turn off the line trimming and the space collapsing first. Indentation is syntax in Python, YAML and Makefiles, and two trailing spaces are a hard line break in Markdown. Removing the zero-width characters stays safe in all of those cases, and it is the option you actually need — a zero-width space pasted into a source file produces an error nobody can see by reading.

For lists, clean before you deduplicate rather than after: two entries that differ by one trailing non-breaking space are two different strings to every deduplication routine ever written. Removing duplicate lines from a list covers the other half of that problem, which is deciding whether case and word order count as a difference.

When should you leave them alone?

A blanket strip is wrong at least as often as it is right.

• Emoji. A single developer emoji is a person, a zero-width joiner and a laptop. Remove the joiner and you get two separate emoji. Family emoji fall apart the same way.
• Persian, Arabic and Indic text. The zero-width non-joiner is not decoration in Persian — it separates parts of a word that must not fuse, and deleting it changes how the word reads. The joiner does the same job in Devanagari.
• Deliberate typography. A non-breaking space between a number and its unit is there to stop a line break splitting "10 km" in half. A cleaner cannot tell that one from an accidental one, and flattens both.

That is the honest limit of every tool in this category: it finds characters, it cannot read intent.

What a whitespace cleaner will not catch

Several invisible-character problems look identical from the outside and need a different fix:

• Homoglyphs. A Cyrillic а and a Latin a render identically and are different characters. No amount of whitespace cleaning makes them match. If a string still will not compare equal after a clean, this is the next suspect.
• The soft hyphen, U+00AD. Invisible until the line breaks there, at which point a hyphen appears. Word and hyphenation tools produce them, and most cleaners leave them alone because they are not spaces.
• Bidirectional control characters, U+202A to U+202E and U+2066 to U+2069. They reorder how text is displayed without changing what it contains. This is the Trojan Source trick: a security problem rather than a tidiness one.
• Line endings. Windows CRLF endings mixed into Unix LF ones make a diff claim every line changed, and no page with a text box can fix it. HTML normalises the contents of a textarea to plain line feeds, so the carriage returns are gone before any script sees them. Fix it in your editor, with dos2unix, or in git's conversion setting.

All four have the same signature as an invisible space, and none of them respond to the same treatment. Working out which one you have is most of the job.

Last updated September 19, 2026