How does Unicode actually store so many characters?

In one line

The rule

Unicode is a standard that assigns a unique number (code point) to every character, no matter what language or symbol it is.

A Unicode code point is written as U+XXXX, where XXXX is a hexadecimal (base-16) number.

Let's understand it simply

Imagine you are running a huge library. Every book needs a special number so you can find it easily. Unicode does the same for letters, symbols, and emojis. It gives each one a unique number, called a code point. For example, the letter 'A' gets U+0041, 'अ' (Hindi) gets U+0905, and '😊' (smiling face emoji) gets U+1F60A.

But computers only understand 0s and 1s (called bits). So, Unicode also tells computers how to turn these numbers into bits, using special rules called encodings (like UTF-8, UTF-16, UTF-32).

How it works

1. Unicode makes a giant list of all characters in all languages and symbols.
2. Each character is given a unique number (code point), like a roll number in school.
3. To store or send these characters, the code point is changed into bits using an encoding.
4. The most common encoding is UTF-8. It uses 1 to 4 bytes (8 bits each) for each character, depending on how big the code point is.
5. When you type or read a character, the computer looks up the code point and shows the correct symbol.
6. This way, the same bits always mean the same character, everywhere in the world.

The story behind it

Before Unicode, computers used many different systems (like ASCII or ISCII) for different languages. These systems clashed—one file might show Hindi letters as garbage on another computer. In 1991, a group of engineers, including Joe Becker, Lee Collins, and Mark Davis, started Unicode to solve this. They wanted a single system for every language. Now, Unicode is managed by the Unicode Consortium, a group of companies and experts who keep adding new characters (like new emojis or scripts).

Where you see it in real life

• WhatsApp messages: You can send Hindi, Tamil, English, or even emojis—all work because of Unicode.
• Websites: Browsers use Unicode to show text in any language, so you can read news in Marathi or Japanese.
• Mobile keyboards: When you switch from English to Hindi or add a smiley, Unicode makes sure the right character appears.
• Programming: When you write code in Python or Java, strings are stored using Unicode, so you can use any language.
• ATM screens: Instructions in multiple languages are possible because of Unicode.
• PDFs and e-books: Unicode lets you read a book in any script, from Sanskrit to Russian.

Why it matters

• Universal communication: Unicode lets people from any country read and write in their own language on computers and phones.
• No mix-ups: The same code always means the same character, so files look right everywhere.
• Supports new symbols: New emojis, scripts, and symbols can be added without breaking old data.
• Needed for programming: Modern software, websites, and apps all rely on Unicode to handle text.

Worked example

Let’s see how Unicode stores the Hindi letter 'क'.

Given: Character = 'क'
Step 1: Find its Unicode code point: U+0915
Step 2: Write U+0915 in binary: 0000 1001 0001 0101
Step 3: Use UTF-8 encoding: 'क' is stored as two bytes: 0xE0 0xA4 0x95
Step 4: When you type 'क', the computer stores these three bytes.
Step 5: Any device reading these bytes with Unicode will show 'क'.

Common mistakes

• Thinking Unicode is an encoding: Unicode is the list of characters and their numbers. UTF-8, UTF-16, etc., are the encodings.
• Believing all characters use the same number of bytes: In UTF-8, some use 1 byte, others use up to 4.
• Assuming Unicode only works for English: It covers almost every language, symbol, and emoji.
• Mixing up code points and bytes: Code point is the number (like U+0915), bytes are how it is stored (like E0 A4 95).

Remember it

Unicode is like a giant dictionary: every character gets its own number, and encodings turn those numbers into computer bits.