What is Unicode and how is it different from UTF-8?

In one line

The rule

Unicode is a universal character encoding standard that assigns a unique number (code point) to every character in every language. UTF-8 is a specific way (encoding) to represent those Unicode code points as a series of 8-bit bytes, so computers can store and send them.

Let's understand it simply

Imagine you have a giant dictionary that lists every letter, symbol, and emoji from every language in the world. This dictionary gives each character its own number. That is Unicode — the master list.

But computers only understand 0s and 1s. So, we need a way to turn those numbers into a form computers can store and share. That is what UTF-8 does. It is like a recipe for turning the Unicode number into a string of bytes (small chunks of data).

How it works

1. Unicode gives every character a unique number, called a code point (like 'A' = 65, 'अ' = 2309).
2. UTF-8 takes this code point and turns it into 1, 2, 3, or 4 bytes (groups of 8 bits).
3. For simple English letters, UTF-8 uses just 1 byte (so old files still work).
4. For other languages or emojis, UTF-8 uses more bytes as needed.
5. When you type or send a message, your device uses UTF-8 to store or send the Unicode code points as bytes.
6. When someone reads the message, their device turns the bytes back into the right characters using UTF-8 rules.

The story behind it

Unicode was started in the late 1980s by a group of computer experts from Apple and Xerox. Before Unicode, every country had its own code for letters (like ASCII for English, ISCII for Indian scripts), and computers could not easily share text in different languages. Unicode solved this by making one big list for everyone.

UTF-8 was created by Ken Thompson and Rob Pike in 1992. They wanted a way to store Unicode characters that worked well with old systems (like those using ASCII) and saved space for common letters.

Where you see it in real life

• WhatsApp messages: When you send 'नमस्ते' or an emoji, Unicode gives each symbol a number, and UTF-8 stores it as bytes.
• Websites: Most websites use UTF-8 to show text in Hindi, Tamil, Chinese, Arabic, and more.
• Programming: When you write code in Python or Java, strings are stored using Unicode, and files are saved as UTF-8.
• File names: You can save a file as 'रोटी.txt' because your computer uses Unicode and UTF-8.
• SMS and emails: These use UTF-8 to send messages in any language.
• Operating systems: Windows, Linux, and Android all use Unicode and UTF-8 for text.

Why it matters

• Global communication: Unicode and UTF-8 let people write and read in any language on any device.
• No data loss: Text does not get garbled when moving between systems or countries.
• Programming: Developers do not have to worry about language support — Unicode covers all.
• Internet: UTF-8 is the most common way to store and send text on the web.

Worked example

Let's see how the Hindi letter 'अ' (Unicode code point U+0905) is stored in UTF-8:

Given: Unicode code point for 'अ' = U+0905 (decimal 2309)
Step 1: Check if code point fits in 1 byte (0–127)? No.
Step 2: UTF-8 uses 3 bytes for this range (U+0800 to U+FFFF)
Step 3: Convert 2309 to binary: 0000 1001 0000 0101
Step 4: Arrange bits in UTF-8 3-byte pattern: 1110xxxx 10xxxxxx 10xxxxxx
Step 5: Fill in the bits: 1110 0000 1010 0000 1000 0101
Step 6: Final bytes (in hex): E0 A4 85

So, 'अ' is stored as three bytes: E0 A4 85 in UTF-8.

Common mistakes

• Thinking Unicode and UTF-8 are the same: Unicode is the list; UTF-8 is one way to store it.
• Assuming UTF-8 uses 1 byte for every character: Only basic English letters use 1 byte; other characters use more.
• Believing all Unicode files are UTF-8: Unicode can be stored in other ways too (like UTF-16, UTF-32).
• Mixing up encoding and font: Unicode is about numbers for characters, not how they look.

Remember it

Unicode is the list of all characters; UTF-8 is the method to store them as bytes.