How does UTF-8 know how many bytes to use for each character?

In one line

The rule

In UTF-8, the number of bytes for a character is decided by the pattern of the first byte.

Here are the patterns:

• If the first bit is 0: 1 byte (ASCII)
• If the first bits are 110: 2 bytes
• If the first bits are 1110: 3 bytes
• If the first bits are 11110: 4 bytes

Let's understand it simply

Imagine you are reading a message in code. Each word starts with a special sound that tells you how many words to listen for. UTF-8 works like this: the first few bits in a byte tell your computer, “Hey! This letter will take 1, 2, 3, or 4 bytes.”

Think of it like a traffic signal. A green light (0) means go ahead, just one byte. A yellow light (110) means slow down, two bytes are coming. Red (1110 or 11110) means even more bytes are needed. The computer reads the signal and knows how many bytes to group together.

How it works

1. The computer reads the first byte of a character.
2. It checks the first few bits (the leftmost bits).
3. If the first bit is 0, it’s a 1-byte character (like English letters).
4. If the first bits are 110, it’s a 2-byte character (like é or आ).
5. If the first bits are 1110, it’s a 3-byte character (like क or 中).
6. If the first bits are 11110, it’s a 4-byte character (like emojis or rare scripts).
7. The following bytes (if any) always start with 10, so the computer knows they are not the start of a new character.

The story behind it

UTF-8 was created in 1992 by Ken Thompson and Rob Pike. They needed a way for computers to handle all the world’s writing systems, not just English, but also Hindi, Chinese, Arabic, and more. Before UTF-8, computers used many different codes, which caused confusion and errors. UTF-8 solved this by using a clever system where the number of bytes is clear from the first byte.

Where you see it in real life

• Websites: Every time you see Hindi, Tamil, or Chinese on a website, UTF-8 is working to show those letters.
• WhatsApp messages: Emojis and different scripts are sent using UTF-8, so your friend sees the same smiley face.
• Programming: When you write code in Python or Java, UTF-8 helps store and read text in any language.
• Text files: Notepad and other editors use UTF-8 to save files so anyone can open them anywhere.
• Databases: Names and addresses in many languages are stored using UTF-8.
• Mobile phones: When you type in Hindi or use emojis, your phone uses UTF-8 to store and send those characters.

Why it matters

• Universal: UTF-8 lets computers talk in any language, so you can read and write in Hindi, Tamil, or even use emojis.
• Compatibility: Old ASCII files (English text) still work, because UTF-8 keeps them as 1 byte per letter.
• Efficient: It saves space for common characters and uses more bytes only when needed.
• Foundation: Many modern apps, websites, and systems depend on UTF-8. It is the backbone of the internet.

Worked example

Let’s see how UTF-8 encodes the letter ‘A’ and the Hindi letter ‘क’:

Example 1: English letter 'A' (Unicode U+0041)
Step 1: 'A' in binary: 01000001
Step 2: First bit is 0 → 1 byte
Step 3: UTF-8 uses just 01000001

Example 2: Hindi letter 'क' (Unicode U+0915)
Step 1: 'क' in binary: 1001000010101 (13 bits)
Step 2: Needs more than 7 bits, so not 1 byte
Step 3: First byte starts with 1110 (for 3 bytes)
Step 4: UTF-8 encoding: 1110xxxx 10xxxxxx 10xxxxxx
Step 5: Fill in the bits for 'क': 1110 1000 1010 1000 1010 0101
Step 6: Bytes: E0 A4 95 (in hexadecimal)

Common mistakes

• Thinking every character is 1 byte: Not true. Only English letters and numbers are 1 byte. Other scripts and emojis use more.
• Ignoring the start bits: If you don’t check the first bits, you can’t know how many bytes to read.
• Splitting a character in the middle: If you cut a UTF-8 character in the wrong place, you get junk text (mojibake).
• Assuming all languages use the same number of bytes: Hindi, Chinese, and emojis often use 3 or 4 bytes.

Remember it

The first bits of a UTF-8 byte are like a signal: they tell you how many bytes to read for each character.