How does a computer know how many bytes to read for each character in UTF-8?

In one line

The rule

In UTF-8, the number of bytes for a character is determined by the pattern of the first byte.

Here’s the rule in simple terms:

• If the first bit is 0, it’s a 1-byte character.
• If it starts with 110, it’s a 2-byte character.
• If it starts with 1110, it’s a 3-byte character.
• If it starts with 11110, it’s a 4-byte character.

Let's understand it simply

Imagine you are reading a line of people at the railway station. Each person holds up fingers to show how many people are in their group. If someone holds up 1 finger, it’s just them. If they hold up 2, it’s them and one more. In UTF-8, each character starts with a special sign (the first few bits) to show how many bytes belong together. The computer looks at the first byte’s sign to know if it should read just this one, or a group of 2, 3, or 4 bytes for the character.

How it works

1. The computer reads the first byte.
2. It checks the first bits (from left):

• If the first bit is 0, this is a 1-byte character (like English letters).
• If the first three bits are 110, this is the start of a 2-byte character.
• If the first four bits are 1110, this is the start of a 3-byte character.
• If the first five bits are 11110, this is the start of a 4-byte character.

1. For multi-byte characters, the next bytes always start with 10 (these are called continuation bytes).
2. The computer reads the number of bytes shown by the pattern, then moves to the next character.
3. It repeats this process for the whole text.

The story behind it

UTF-8 was created by Ken Thompson and Rob Pike in 1992. They wanted a way to store all the world’s characters (like Hindi, Tamil, Chinese, emojis) using a system that would still work with old English-only files. Their clever idea was to use special patterns at the start of each character, so computers could always tell where one character ends and the next begins, even if some characters are longer.

Where you see it in real life

• Web pages: Every time you see Hindi, Tamil, or emojis on a website, UTF-8 is working behind the scenes to show those characters.
• WhatsApp messages: When you send a message with a smiley or a non-English word, UTF-8 helps your phone understand how many bytes to read for each character.
• Text files: If you open a .txt file with both English and Hindi, the computer uses UTF-8 rules to display the text correctly.
• Programming: When a Python or Java program reads or writes text, it uses UTF-8 to handle all kinds of characters.
• Databases: MySQL and other databases use UTF-8 to store names and addresses from around the world.
• Operating systems: Linux, Windows, and Android all use UTF-8 to manage file names and user data.

Why it matters

• Universal language support: UTF-8 lets computers handle all languages and symbols in one system.
• Backward compatibility: Old English-only files still work because 1-byte UTF-8 is the same as old ASCII.
• Efficient storage: English text uses just 1 byte per letter, saving space.
• Error detection: If a byte does not match the UTF-8 pattern, the computer knows something is wrong.

Worked example

Let’s see how the Hindi letter 'अ' (U+0905) is stored in UTF-8:

Step 1: Find the Unicode code point for 'अ' → U+0905 (decimal 2309)
Step 2: See which range it falls in → needs 3 bytes (because it is above 2047)
Step 3: Convert 2309 to binary → 100100000101
Step 4: Fill into UTF-8 3-byte pattern: 1110xxxx 10xxxxxx 10xxxxxx
Step 5: Place the bits:
  - 1110 1001 1000 0000 1000 0101 → E0 A4 85 (in hexadecimal)
Step 6: Computer reads E0 → sees 1110, knows it’s a 3-byte character, reads next two bytes (A4, 85)

Common mistakes

• Thinking every character is 1 byte: Many students think all characters are 1 byte, but Hindi, Chinese, and emojis use more.
• Not checking the first bits: Some forget that the first bits tell the length, not the value of the character.
• Mixing up continuation bytes: Only the first byte tells the length; the next bytes always start with 10.
• Assuming all Unicode is UTF-8: Unicode is the big list of characters; UTF-8 is just one way to store them.

Remember it

"Look at the first bits—UTF-8 tells you how many bytes to read!"