creatorvalet

What is Unicode?

Unicode assigns a number to every character in every writing system. UTF-8 decides how those numbers become bytes. Confusing the two causes most text bugs.

Updated 2026-08-04

Unicode is a catalogue. It assigns a unique number — a code point — to every character in every writing system humans use, plus a great many symbols that are not writing at all. The letter A is code point 65. The letter å is 229. The waving hand emoji is 128,075.

That is all Unicode does. It says which number means which character. It says nothing about how those numbers are stored in a file, and that second question is where nearly every text bug lives.

Unicode is not UTF-8

This distinction is the one worth internalising.

Unicode is the mapping from characters to numbers. UTF-8 is one way of turning those numbers into bytes. UTF-16 is another. They describe the same characters and produce completely different files.

UTF-8 is variable-width: plain English letters take one byte each, most European accented letters take two, most Asian characters take three, and emoji take four. That design is why UTF-8 won — an English text file in UTF-8 is byte-identical to the same file in ASCII, so thirty years of existing files kept working.

It is also why å becomes Ã¥ when a file is read with the wrong assumption. Those are its two bytes, interpreted one at a time.

Code points are not characters, either

Here the catalogue gets subtle in ways that break code.

A family emoji looks like one character. It is four people plus three invisible joining characters — seven code points that a font renders as a single picture. A flag is two letters from an alphabet that exists only to make flags. An é might be one code point, or it might be e followed by a combining accent, and both look identical on screen while comparing as different strings.

What a reader perceives as one character is called a grapheme cluster. Counting those correctly is why a properly built character counter reports a family emoji as one character while a naive one reports five — and why removing emoji has to match whole sequences rather than individual code points.

The invisible characters

Unicode includes characters that render as nothing at all: zero-width spaces, non-breaking spaces, soft hyphens, byte order marks, and directional marks. Every one of them has a legitimate use, and every one of them causes confusion when it appears where it was not expected.

A zero-width space pasted into a CSV file breaks the column silently. A non-breaking space in code looks exactly like a normal space and fails to compile. These are not exotic edge cases — they arrive routinely from word processors and web pages.

Homoglyphs

Different code points can look identical. Latin а and Cyrillic а are different characters that most fonts render the same way. This is the mechanism behind homoglyph phishing: a domain name that reads as apple.com but is not.

It is also a routine nuisance in ordinary data, where a name copied from one system will not match the same name in another because one of them contains a Cyrillic letter nobody typed deliberately.