Unicode — A Number for Every Character

In the 1990s an email from Moscow to Delhi often arrived as gibberish. Every region had its own way to squeeze its alphabet into 256 byte values, and the same byte meant different letters in each. Unicode fixed this by giving up on squeezing: it gives every character in every script its own number, once, for everyone.

The rule

Unicode gives every character one number, its code point, written U+hex.

A code point is written U+ followed by at least four hex digits:

Three facts to hold on to:

• The first 128 code points are ASCII. U+0041 is A, exactly as in ASCII, so ASCII text is already valid Unicode.
• The space runs from U+0000 to U+10FFFF, room for 1,114,112 code points. Most are still unassigned.
• A code point is a number, not bytes. How the number is written to memory or a file is a separate choice, called an encoding (UTF-8, UTF-16, UTF-32). The same text can be encoded different ways.

One more subtlety: what a reader sees as one character is not always one code point. é can be the single code point U+00E9, or e followed by a combining accent U+0301. Many emoji, including most flags and family emoji, are several code points joined together. So "string length" depends on what you count.

The shortcut

Keep two layers apart in your head: character → code point (Unicode's job), then code point → bytes (the encoding's job). Whenever text looks wrong, ask which layer broke.

Drills

Answer fast — trust the rule.

Remember

• Unicode gives every character one code point, written U+hex, from U+0000 to U+10FFFF.
• The first 128 code points are ASCII.
• The trap: a code point is not bytes, and one visible character can be several code points.