How does UTF-8 store characters from different languages?

In one line

The rule

UTF-8 is a variable-length character encoding for Unicode, using 1 to 4 bytes for each character.

There is no single formula, but the main idea is:

• 1 byte for basic English (ASCII) characters
• 2, 3, or 4 bytes for other characters (like Hindi, Chinese, emojis)

Let's understand it simply

Imagine you are packing different-sized gifts into boxes. Small gifts (like a pencil) need only a tiny box. Big gifts (like a cricket bat) need a bigger box. UTF-8 works the same way. Simple English letters (A, B, C) fit in a small box (1 byte). More complex letters (like 'अ' or '你') need bigger boxes (2, 3, or 4 bytes).

Think of a WhatsApp message. If you type only English, each letter uses 1 byte. If you type in Hindi or send an emoji, those need more bytes. This way, UTF-8 can store any language or symbol in the same message.

How it works

1. Each character in Unicode has a unique number (code point).
2. UTF-8 checks the code point and decides how many bytes are needed.
3. If the code point is small (0–127), it uses 1 byte (same as old ASCII).
4. If it is bigger, it uses 2, 3, or 4 bytes, with special patterns to show how many bytes are used.
5. The first bits of each byte tell the computer if it is a single-byte or part of a multi-byte character.
6. When reading, the computer looks at the first byte, sees how many bytes to read, and puts them together to get the character.

The story behind it

Unicode was created in the late 1980s and early 1990s because computers in different countries could not share text easily. Each country had its own code for letters, so a Hindi file might look like nonsense in Japan. UTF-8 was invented by Ken Thompson and Rob Pike in 1992. They wanted a way to store all languages using bytes, without breaking old English text files.

A fun fact: UTF-8 was designed so that old English files (ASCII) would still work, but now you could also write in Tamil, Arabic, or even use emojis!

Where you see it in real life

• Web pages: All modern websites use UTF-8 so you can see text in any language.
• WhatsApp and SMS: Messages in Hindi, Bengali, Tamil, or with emojis use UTF-8 to store and send them.
• Programming: Source code files (like Python, Java) use UTF-8 so you can write comments in your own language.
• File names: You can save files with names in Marathi or Japanese because the computer uses UTF-8.
• Databases: Banks and apps store names in any script (like Singh, 李, أحمد) using UTF-8.
• Operating systems: Menus and settings can appear in your local language, thanks to UTF-8.

Why it matters

• Universal communication: Lets people from any country read and write in their own language on the same computer or website.
• No data loss: Old systems lost letters or showed "?" for unknown symbols. UTF-8 keeps all characters safe.
• Supports emojis: All those smileys and flags you use are stored using UTF-8.
• Needed for programming: If you want to make apps for India or the world, you must know UTF-8.

Worked example

Let's see how UTF-8 stores three characters:

Given: Characters = 'A', 'अ', '😊'

Step 1: Find Unicode code points
  'A' = U+0041
  'अ' = U+0905
  '😊' = U+1F60A

Step 2: Find UTF-8 encoding
  'A': U+0041 is 65 in decimal, fits in 1 byte: 01000001
  'अ': U+0905 is 2309 in decimal, needs 3 bytes: 0xE0 0xA4 0x85
  '😊': U+1F60A is 128522 in decimal, needs 4 bytes: 0xF0 0x9F 0x98 0x8A

Step 3: Summary
  'A' = 1 byte
  'अ' = 3 bytes
  '😊' = 4 bytes

Common mistakes

• Thinking every character uses 1 byte. (Wrong: Hindi, Chinese, emojis use more.)
• Mixing up UTF-8 with Unicode. (Unicode is the list of all characters; UTF-8 is how we store them.)
• Forgetting that file size grows if you use lots of emojis or non-English letters.
• Assuming all computers use UTF-8 by default. (Some old systems use other encodings.)

Remember it

UTF-8 is like a smart box: it uses just enough space for each character, so you can write in any language.