What is UTF-8 and how does it work?

In one line

The rule

UTF-8 (Unicode Transformation Format – 8-bit) is a standard for encoding every character in the Unicode system using 1 to 4 bytes.

There is no single formula, but the main rule is:

• English letters and numbers use 1 byte.
• Other characters (like Hindi, Chinese, emojis) use 2, 3, or 4 bytes.

Let's understand it simply

Imagine you want to write a letter to a friend, but your friend only understands numbers. You need a system to turn every letter, symbol, or emoji into a number. That’s what encoding means.

UTF-8 is like a codebook. It tells the computer, “A” is 65, “अ” is 2309, “😊” is 128522, and so on. But computers store everything as 0s and 1s. UTF-8 breaks these numbers into small packets called bytes (a byte is 8 bits, like 8 switches that can be on or off).

UTF-8 is smart: it uses just 1 byte for simple English letters, but more bytes for other languages or symbols. This saves space and lets people all over the world use their own scripts.

How it works

1. Every character in the world has a unique number in Unicode (the global list of characters).
2. UTF-8 takes that number and turns it into 1 to 4 bytes.
3. If the character is simple (like A–Z, 0–9), it uses only 1 byte.
4. If it’s a special symbol or from another language, it uses 2, 3, or 4 bytes.
5. When you open a file, the computer reads the bytes and turns them back into letters using the UTF-8 codebook.

The story behind it

In the early days, computers in America used ASCII, which could only store English letters and a few symbols. But as computers spread to other countries, people needed to write in Hindi, Tamil, Chinese, Arabic, and more. Different countries made their own systems, but these did not work well together.

In the 1990s, the Unicode Consortium (a group of experts) created Unicode: a single list for every character in every language. UTF-8 was invented by Ken Thompson and Rob Pike in 1992 to store these characters efficiently and safely, so files could be shared worldwide without confusion.

Where you see it in real life

• WhatsApp messages: When you send a message in Hindi, English, or with emojis, UTF-8 makes sure it looks right on your friend’s phone.
• Websites: Most websites use UTF-8 so people can read them in any language.
• File names: You can save a file as "रोटी.jpg" or "smile😊.png" because of UTF-8.
• Programming: When you write code in Python or Java, UTF-8 lets you use any language in your strings.
• Mobile apps: Apps like Instagram or Facebook use UTF-8 to show posts from all over the world.
• Government forms: Online forms in India support names in regional scripts because of UTF-8.

Why it matters

• Universal communication: Lets people write and read in any language on any device.
• Saves space: English text uses less memory, but other scripts are supported too.
• Prevents errors: Stops garbled text (like "ä") when sharing files between countries.
• Essential for the internet: Almost every website, app, and email uses UTF-8.

Worked example

Let’s encode the letter "A" and the Hindi letter "अ".

Given: "A" (Unicode number 65), "अ" (Unicode number 2309)
Step 1: "A" is in the basic English set, so UTF-8 uses 1 byte
Step 2: 65 in binary is 01000001
Step 3: "अ" is outside the basic set, so UTF-8 uses 3 bytes
Step 4: 2309 in binary is 0000 1001 0000 0101
Step 5: UTF-8 splits this into 3 bytes: 0xE0 0xA4 0x85
Step 6: Computer stores these bytes; when reading, it knows how to join them back to "अ"

Common mistakes

• Thinking UTF-8 is only for English. It supports every language.
• Believing every character takes the same space. English letters use 1 byte, but others may use more.
• Mixing up UTF-8 and Unicode. Unicode is the list of characters; UTF-8 is how to store them.
• Not setting UTF-8 in a file or website, leading to broken or strange-looking text.

Remember it

UTF-8 = "One world, one code for all letters and emojis."