Chapter 5 — Working With Data In Python

What is data? Data are pieces of information — numbers, text, True/False values or more complex records — that describe events, measurements, observations or transactions.

Why work with data? In computing and everyday life we collect, store, analyze and present data to make decisions (e.g., student marks, sales figures, weather records).

Types of data:

• Primitive: integers, floats, strings, booleans.
• Structured: tables (rows and columns), CSV, JSON; unstructured: free text, images.

Common Python containers: list (ordered collection), tuple (ordered immutable), dict (key→value mapping), set (unique items). These are used to store and manipulate data in memory.

Typical data workflow: collection → cleaning/preprocessing → storage → analysis → visualization → interpretation. Cleaning includes handling missing values, converting data types, removing duplicates and validating ranges.

Basic operations: indexing/slicing, iterating (for-loops, comprehensions), aggregations (sum, count, min, max), and summary statistics (mean, median, mode, variance). For files and exchange formats you use CSV or JSON; Python has built-in csv/json modules and higher-level libraries such as pandas for tabular data.

Practical tips: always check data types, handle missing or invalid entries before analysis, and start with simple visualizations to understand distribution and trends.

What is a Variable?
A variable is a name that refers to a value stored in memory. In Python a variable name (identifier) is bound to an object; the object has a value and a type. Python is dynamically typed: the same variable name can refer to objects of different types at different times.

What is an Identifier?
An identifier is the name used to identify a variable, function, class, module, or other object. Identifiers in Python are sequences of letters (a–z, A–Z), digits (0–9) and underscores (_), but must start with a letter or underscore.

Rules for Identifiers

• Must begin with a letter (a–z, A–Z) or underscore (_).
• Can contain letters, digits and underscores only.
• Case-sensitive: age and Age are different identifiers.
• Cannot be a Python reserved keyword (for, if, while, def, class, True, None, etc.).
• Should not start with digits and should not contain special characters or spaces.

Naming Conventions (Good Practices)
Use meaningful names, prefer snake_case for variables and functions (example: student_marks), limit length to readable size, avoid single-letter names except for counters (i, j) and avoid names that shadow built-ins (like list, str).

How Variables Work (Conceptual Memory Model)
Think of a variable name as a label attached to an object stored in memory. When you assign x = 10, the name x points to an integer object with value 10. Assigning x = 'hello' rebinds the name x to a string object; the old integer object may be reclaimed by garbage collection if no names refer to it.

Scope and Lifetime (Brief)
Scope determines where an identifier is accessible: global scope (module-level), local scope (inside functions), and nested scopes. Lifetime is how long the binding exists — local variables exist during function execution; globals exist while the program runs (or module is loaded).

Common Operations

# assignment
count = 5

# multiple assignment
a, b = 1, 2

# swapping without temp
a, b = b, a

# augmented assignment
count += 1

# type conversion (casting)
age = int('18')

Errors to Watch For

• SyntaxError if identifier starts with digit or has illegal character: 2name, first-name.
• NameError when using an identifier that has not been defined.
• Shadowing built-ins can cause subtle bugs (for example, assigning list = [1,2] then calling list()).

Data types specify what kind of value a variable can hold and what operations are valid on it. In Python, values are created by writing literals — fixed notations for data (like 10, 'abc', [1,2]).

Main categories:

• Numeric types
  • int – integers without fractional part (e.g., 42, -7).
  • float – floating-point numbers with decimals (e.g., 3.14, -0.5).
  • complex – complex numbers with real and imaginary parts (e.g., 2+3j).
• Text
  • str – sequence of characters. Literals can use single, double or triple quotes: 'hi', "hello", '''multi-line'''. Supports indexing, slicing and many methods.
• Boolean – bool which has two literals: True and False. Used for conditions and logical operations.
• None – the None literal represents absence of a value.
• Collection (compound) types
  • list – ordered, changeable sequence written with square brackets: [1, 2, 3] (mutable).
  • tuple – ordered, fixed sequence written with parentheses: (1, 2, 3) (immutable).
  • set – unordered collection of unique elements written with braces: {1, 2, 3} (mutable, no duplicates).
  • dict – key:value mapping using braces: {'name':'Asha', 'age':16} (mutable, keys unique).

Literals are the concrete notation for values. Examples: numeric literals (0, -3.5), string literals ("CBSE"), boolean (True), collection literals ([], (), {}), and None.

Mutability: lists, sets and dicts are mutable (their contents can change). Strings, tuples, integers, floats and complex numbers are immutable. Understanding immutability helps avoid bugs when sharing objects.

Type checking and conversion: use type(x) or isinstance(x, T) to check types. Convert values with int(), float(), str(), bool(), and constructors for collections like list(), tuple(), set(), dict().

Why it matters (real life): choose appropriate types to represent data accurately — e.g., counts as int, prices/measurements as float, names as str, a contact directory as dict. Wrong type choices lead to incorrect computations or inefficient programs.

Type conversion (also called casting) in Python is the process of converting a value from one data type to another. This is important when combining values of different types, reading input (which is a string), or preparing data for operations that require specific types.

Two kinds of conversion

• Implicit conversion (automatic): Python automatically converts one data type to another when it is safe to do so—for example, when an int and a float are used together, the int is converted to float so precision is preserved.
• Explicit conversion (casting): The programmer converts types using built-in functions such as int(), float(), str(), bool(), etc. This is required when automatic conversion is not possible or when you want a specific result.

Rules & behavior

• Numeric promotion: int mixed with float → float; float mixed with complex → complex.
• int(float_value) truncates toward zero (it does not round).
• Converting from string: the string must represent a valid value for the target type (e.g., "123" → int OK, "12.3" → int raises ValueError unless first converted to float).
• bool conversions: bool(0) is False; bool(0.0) is False; bool("") is False; any nonzero number or non-empty container is True.
• Incompatible conversions (like int('abc')) raise ValueError; type mismatches in operations may raise TypeError.
• Python’s bool is a subclass of int: True == 1 and False == 0, so booleans participate in numeric operations.

Common casting functions

• int(x) — to integer (truncates floats).
• float(x) — to floating-point.
• str(x) — to string.
• bool(x) — to boolean.
• complex(a, b) or complex(x) — to complex number.
• int(string, base) — convert from a string in given base (e.g., base=2 for binary).
• bin(x), oct(x), hex(x) — convert integer to binary/octal/hex string representation.
• list(), tuple(), set(), dict() — convert between container types where conversion makes sense.

Short code examples

# implicit conversion
result = 5 + 2.3    # result is 7.3 (int 5 promoted to float)

# explicit conversion
s = '123'
num = int(s)         # 123 (int)
price = 19.95
count = int(price)   # 19 (truncated)

# string + number needs casting
name = 'Alice'
age = 17
greeting = name + ' is ' + str(age) + ' years old'

# base conversion
n = int('1010', 2)   # n == 10

# boolean rules
bool(0)    # False
bool('')   # False
bool([1])  # True

# invalid conversion
# int('12.3')  -> ValueError; first do float('12.3') then int(...)

Understanding when Python converts types automatically and when you must cast explicitly helps avoid errors and data loss (for example, unintended truncation). Use explicit casting when you need a specific type or to validate/clean input data.

Overview
In Python, an expression is any combination of values, variables and operators that Python can evaluate to produce another value. Operators are special symbols or keywords that perform operations on one or more operands (values or variables). Understanding operators and expressions is essential for writing computations, making decisions and manipulating data.

Types of operators

• Arithmetic: +, -, *, /, // (floor division), % (modulus), ** (exponent). These perform numeric calculations. Example: 3 + 4 * 2**2.
• Comparison (Relational): ==, !=, <, <=, >, >=. They compare values and return True or False.
• Logical: and, or, not. Combine boolean expressions. Use short-circuit evaluation.
• Assignment: = assigns a value. Augmented assignment like +=, -=, *= updates a variable: x += 3 is x = x + 3.
• Bitwise: &, |, ^, ~, <<, >> operate on integer bit representations.
• Membership: in, not in — check presence in sequences or collections.
• Identity: is, is not — check whether two names refer to the same object (not just equal values).

Expressions and evaluation
An expression can be simple (5, x) or compound ((a + b) * c). Evaluation follows operator precedence (higher-precedence operators evaluated first) and associativity (left-to-right or right-to-left for operators with same precedence). Python also performs implicit type conversions for some operations (e.g., 1 + 2.0 -> 3.0) and allows explicit conversions via int(), float(), str(), bool().

Precedence highlights
Important precedence order (from higher to lower): parentheses, exponentiation (** right-to-left), unary +/-, *, /, //, %, +, -, bitwise shifts, bitwise AND/OR/XOR, comparison, not, and, or. Use parentheses to force a specific evaluation order.

Short-circuiting
Logical and and or use short-circuit evaluation: for A and B, if A is false, B is not evaluated; for A or B, if A is true, B is not evaluated. This is useful for guarding operations (e.g., x and x.method()).

Operator overloading
Some operators behave differently for different operand types: e.g., + adds numbers but concatenates strings or lists ('a' + 'b' -> 'ab', [1] + [2] -> [1,2]).

Common pitfalls

• Using is instead of == for value comparison (use == to compare values; is checks identity).
• Unexpected integer division vs floor division: in Python 3, / produces float, use // for floor division.
• Relying on implicit type conversion in mixed-type expressions can produce unexpected floats or strings; prefer explicit conversion when needed.

Small Python code examples

# arithmetic and precedence
a, b, c = 2, 3, 4
result = a + b * c        # 2 + (3*4) = 14

# short-circuit example
user = None
name = user and user.name  # user is None -> name is None (user.name not evaluated)

# bitwise example
x = 6        # 0b110
y = 3        # 0b011
z = x & y    # 0b010 -> 2

# membership
s = 'hello'
'in' in s    # True

# augmented assignment
count = 0
count += 5   # 5

# swapping without temp
x, y = y, x

How this applies in real life
Operators and expressions model everyday computations: banking calculations (interest using arithmetic operators and precedence), decision logic in apps (comparison + logical operators to determine eligibility), string manipulation in text processing (concatenation, membership), bitwise operations in low-level hardware control, and short-circuit checks for safe resource access.

What is a string?
A string is an ordered sequence of characters used to store and manipulate text. In Python a string literal is written between single quotes (') or double quotes (") or triple quotes for multi-line text. Strings are immutable — once created, their characters cannot be changed in place.

Creating and representing strings

• s = 'Hello' or s = "Hello" or s = '''Multi\nline'''
• Escape sequences: \n (newline), \t (tab), \\ (backslash), \', \"
• Raw strings: r'C:\folder\file' treats backslashes literally

Indexing and slicing
Strings are indexed from 0. You can access characters and sub-strings using indexing and slicing.

• Indexing: s[i] gives the character at position i (negative indices count from the end)
• Slicing: s[i:j:k] returns a substring from index i up to j (exclusive) with step k; omitted values default to start, end, or step 1

Common operations

• Concatenation: s + t
• Repetition: s * n
• Membership: sub in s checks whether sub occurs in s
• Length: len(s)
• Methods: s.upper(), s.lower(), s.strip(), s.split(sep), sep.join(list), s.replace(old,new), s.find(sub), s.count(ch)

Immutability and consequences
Because strings are immutable, operations that appear to modify a string actually create and return a new string. For example, s = s.replace('a','b') assigns a new string to the variable s; the old string is unchanged in memory until garbage-collected.

Formatting and templates
Strings are commonly used to format output and generate templates. You can use str.format(), f-strings (f"Hello {name}"), or % formatting to insert values into text.

Use in data handling
Strings are central in reading/writing text files, parsing CSV or log lines, processing user input, building URLs and JSON, validating input (emails, phone numbers), and producing human-readable reports.

What is a list?

A list in Python is an ordered, mutable collection of items. Items can be of different data types (heterogeneous) and duplicates are allowed. Lists are used to store a sequence of values and provide many built-in operations and methods to manipulate that sequence.

Creating lists

# empty list
lst = []
# list with values
nums = [10, 20, 30]
mixed = [1, 'apple', 3.14, True]
# nested list (list of lists)
matrix = [[1,2,3], [4,5,6], [7,8,9]]

Key characteristics

• Ordered: items have a definite order and each item has an index (starting at 0).
• Mutable: elements can be changed, added or removed after creation.
• Heterogeneous: elements of different types can coexist.
• Allow duplicates: same value can appear multiple times.

Indexing and slicing

Access items by index: lst[0] is first item. Negative indices count from the end: lst[-1] is last item. Slicing returns a sublist: lst[start:stop:step].

nums = [10,20,30,40,50]
nums[0]      # 10
nums[-1]     # 50
nums[1:4]    # [20,30,40]
nums[::2]    # [10,30,50]

Common list operations and methods

• len(lst) — number of elements.
• in — membership test: if x in lst:.
• Concatenation: lst1 + lst2. Repetition: lst * n.
• append(x) — add x to end.
• insert(i, x) — insert x at index i.
• extend(iterable) — add all elements of an iterable.
• remove(x) — remove first occurrence of x.
• pop([i]) — remove and return element at index i (default last).
• index(x), count(x) — locate/count occurrences.
• sort() and reverse() — in-place sorting and reversal. sorted(lst) returns a new sorted list.
• copy() or slicing lst[:] — shallow copy. Use copy.deepcopy() for nested lists.

Iteration and enumeration

for item in lst:
    print(item)
# with index
for i, v in enumerate(lst):
    print(i, v)

Nested lists (2D lists / matrices)

Lists can contain other lists. Access by chained indices: matrix[row][col]. Useful to represent tables or matrices.

List comprehensions (concise construction)

squares = [x*x for x in range(1,6)]  # [1,4,9,16,25]
filtered = [x for x in nums if x > 20]

Practical tips

• Use append for incremental building, extend to merge lists.
• Prefer sorted() if you need original list unchanged.
• Beware of aliasing: b = a makes both names refer to same list; use a.copy() or a[:] to copy.

What is a tuple?

A tuple is an ordered, immutable collection of items in Python. Like a list, a tuple can hold heterogeneous elements (different data types) and preserves the insertion order, but once created its elements cannot be changed (no item assignment, insertion, or deletion). Tuples are written with parentheses () or simply comma-separated values.

Syntax & basic examples

# creation
t1 = (1, 2, 3)
# parentheses optional for simple tuples
t2 = 4, 5, 6
# single element tuple needs a trailing comma
single = (10,)
# empty tuple
empty = ()

Properties

• Ordered: elements have a fixed order and can be accessed by index (0-based).
• Immutable: you cannot change elements after creation. Any attempt to assign to an index raises an error.
• Heterogeneous: elements of different types allowed.
• Hashable (sometimes): a tuple is hashable if all its elements are hashable, so it can be used as a dictionary key.
• Supports many sequence operations: indexing, slicing, concatenation, repetition, membership test, length, iteration.

Common operations (with examples)

t = (10, 20, 30, 'a')
# indexing
x = t[1]        # 20
# slicing
s = t[1:3]      # (20, 30)
# concatenation
t3 = t + (100,)
# repetition
r = t * 2       # (10,20,30,'a',10,20,30,'a')
# membership
is_in = 20 in t # True
# length
n = len(t)      # 4
# methods: count and index
cnt = t.count(20)
pos = t.index('a')
# unpacking
a, b, c, d = t
# extended unpacking
first, *middle, last = (1,2,3,4,5)

Immutability explained

Because tuples are immutable, operations that seem to change a tuple actually create a new tuple. For example concatenation t + u returns a new tuple; converting to a list, modifying, then converting back is a common workaround.

t = (1,2,3)
# this raises TypeError
# t[0] = 10
# workaround
lst = list(t)
lst[0] = 10
t = tuple(lst)

When to use tuples (advantages)

• When data should not change (fixed records like coordinates, RGB colors, date components).
• As keys in dictionaries (if elements are hashable) or elements in sets.
• Returning multiple values from a function (function returns a tuple by default).
• Tuples are slightly faster and use less memory than lists for fixed-size sequences.

Examples of real-life mapping

• Geographic coordinate: (latitude, longitude)
• RGB color: (R, G, B)
• Date: (year, month, day)
• Database record row (id, name, age)

Notes for Class 11 students

• Understand difference between tuple and list: mutability is the key difference.
• Learn tuple packing/unpacking and common operations (indexing, slicing, concatenation, repetition, membership).
• Practice with small programs: returning multiple values, using tuples as keys in dicts, and converting between list & tuple.

What is a set? A set is an unordered collection of distinct elements. In Python, sets store unique items and support mathematical set operations (union, intersection, difference, etc.). Sets are mutable (you can add/remove elements) but their elements must be immutable (e.g., numbers, strings, tuples).

How to create sets in Python

• Using curly braces: {1, 2, 3}.
• Using the constructor: set([1, 2, 3]) or set('abc').
• Note: {} creates an empty dictionary — use set() for an empty set.

Important properties

• Uniqueness: duplicates are automatically removed.
• Unordered: no guaranteed index or order.
• Mutable container: methods like add, remove, discard, pop, clear are available.
• Set elements must be hashable (immutable types).

Common operations and methods

• Union: A | B or A.union(B) — elements in A or B.
• Intersection: A & B or A.intersection(B) — elements in both A and B.
• Difference: A - B or A.difference(B) — elements in A but not in B.
• Symmetric difference: A ^ B or A.symmetric_difference(B) — elements in A or B but not both.
• Subset / superset checks: A.issubset(B), A.issuperset(B).
• Membership: x in A (fast, average O(1)).

Why use sets? They are ideal for removing duplicates, membership testing, and performing mathematical set operations efficiently.

Simple Python example (illustration)

students = ['Anita', 'Ravi', 'Anita', 'Sunil']
unique = set(students)   # {'Anita', 'Ravi', 'Sunil'}
unique.add('Meera')
if 'Ravi' in unique:
    unique.remove('Ravi')

What is a Dictionary?

A dictionary in Python is a built-in collection type that stores data as key:value pairs. Each key maps to a value. Keys must be unique and immutable (for example: strings, numbers, tuples), while values can be any Python object (numbers, strings, lists, other dictionaries, etc.).

Characteristics

• Mapping type — data accessed by key, not by index.
• Keys are unique; assigning the same key again updates the value.
• Mutable — you can add, change or remove items.
• Insertion order is preserved in Python 3.7+ (implementation detail in 3.6).

Basic operations and syntax

• Creation: d = {'a': 1, 'b': 2} or d = dict().
• Access: value = d['a'] (raises KeyError if key missing) or value = d.get('a', default) (safer).
• Insert / Update: d['c'] = 3.
• Delete: del d['b'] or d.pop('b', default) or d.popitem().
• Membership: 'a' in d returns True/False (checks keys).
• Iteration: for k in d: or for k, v in d.items():.

Common methods

• d.keys() — view of keys.
• d.values() — view of values.
• d.items() — view of (key, value) pairs.
• d.update(other) — merge another dict (overwrites on same keys).
• d.clear() — remove all items.
• d.copy() — shallow copy.
• dict.fromkeys(seq, value) — create dict with keys from seq and same value.
• d.setdefault(key, default) — get or set default value for key.

Dictionary comprehensions

Similar to list comprehensions: {k: v for k, v in iterable}. Useful to build transformed dictionaries in one expression.

Nested dictionaries

Dictionaries can contain other dictionaries as values, enabling structured records (e.g., student -> {name, marks}). Access uses multiple key lookups: students['s1']['marks'].

Performance

Dictionaries are implemented with hash tables. Average time complexity for lookup, insert and delete is O(1). In rare worst-case situations (many hash collisions) complexity can degrade toward O(n).

When to use a dictionary?

• When you need a lookup by a unique key (e.g., phone number by name).
• When representing records/objects with named fields.
• When counting/frequency mapping (word counts, item counts).

Overview: In Python a sequence is an ordered collection of items. Common sequence types are strings, lists and tuples. Sequences support indexing (accessing single elements), slicing (extracting subsequences), and several operations such as concatenation, repetition and membership tests.

Indexing

• Indices are zero-based: the first element has index 0, the second index 1, and so on. Example: seq[0] is the first element.
• Negative indices access elements from the end: seq[-1] is the last element, seq[-2] is the second last, etc. Formula: seq[-k] == seq[len(seq) - k].
• Attempting to use an index outside the valid range raises IndexError for lists/tuples/strings.

Slicing

• General form: seq[start:stop:step]. The slice returns a new sequence of the same type (for strings, lists, tuples) containing elements from index start up to but not including stop, taken every step positions.
• Defaults: if start is omitted it defaults to 0 (when step > 0). If stop is omitted it defaults to len(seq). If step is omitted it defaults to 1.
• Negative step reverses direction: seq[::-1] returns the sequence reversed. With a negative step, default start becomes len(seq)-1 and default stop becomes -1 (stop excluded).
• Slices always produce a new object (a shallow copy for lists). For immutable sequences like strings and tuples you cannot modify elements; for lists you can assign to a slice to change multiple elements at once, e.g. lst[1:3] = [x, y].

Common sequence operations

• Concatenation: seq1 + seq2 joins two sequences of the same type (strings or lists or tuples).
• Repetition: seq * n repeats the sequence n times.
• Membership: x in seq returns True if x appears in seq.
• Length and extrema: len(seq), min(seq), max(seq) (when elements are comparable).
• Iteration: for loops iterate elements in order: for item in seq:

Practical notes and pitfalls

• Remember that slicing stop index is exclusive: seq[1:4] returns elements at indices 1,2,3.
• Using an out-of-range index for slicing does not raise an error; Python clamps the indices to valid range. But direct indexing with an out-of-range index raises IndexError.
• Because slicing returns a new object, modifying the returned slice does not change the original sequence (except by assigning back to a slice of a mutable sequence).

Short code summary

s = 'python'
# indexing
s[0]     # 'p'
s[-1]    # 'n'
# slicing
s[1:4]   # 'yth'  (indices 1,2,3)
s[:3]    # 'pyt'
s[3:]    # 'hon'
s[::2]   # 'pto'  (every second char)
s[::-1]  # 'nohtyp' (reversed)

lst = [10,20,30,40,50]
lst[1:4]         # [20,30,40]
lst[1:4:2]       # [20,40]
lst[-3:-1]       # [30,40]
lst + [60,70]    # [10,20,30,40,50,60,70]
lst * 2          # [10,20,30,40,50,10,20,30,40,50]

Overview

In Python, every value is an object. Objects are either mutable (their content can change after creation) or immutable (their content cannot change; any operation that looks like a change actually creates a new object). Aliasing happens when two or more names (variables) refer to the same object in memory.

Why this matters

• Mutability affects program behaviour: modifying a mutable object through one name affects all names that refer to it.
• Immutability gives safety and predictability: you cannot accidentally change an object shared elsewhere.
• Aliasing can cause bugs if you unintentionally share and then mutate objects.

Mutable vs Immutable (common Python types)

• Mutable: list, dict, set, bytearray, most custom objects (unless designed otherwise)
• Immutable: int, float, bool, str, tuple, frozenset, bytes

Identity vs Equality

• Equality (==) checks whether values are equal.
• Identity (is) checks whether two names point to the same object (same memory address).
• id(x) returns the unique identity (address) of object x while it exists.

Typical behaviours

• Assigning a name to an object just creates a reference (no copy). Example: a = [1, 2]; b = a — both a and b reference the same list.
• Mutating methods change the object in-place: append, extend, pop, clear, dictionary updates, etc.
• Operations on immutables produce new objects. For strings and tuples, operations that appear to change them actually bind the name to a new object.

Shallow vs Deep Copy

• Shallow copy copies the container but keeps references to the same elements (use list.copy(), slicing [:], or copy.copy()).
• Deep copy recursively copies nested objects so no shared sub-objects remain (use copy.deepcopy()).

Practical advice

• When you need independent objects, explicitly copy (shallow or deep) instead of relying on assignment.
• Use immutable objects for keys in dictionaries (e.g., tuple but not list).
• Use is only to check identity (such as None checks); use == for value equality.

Short code demonstrations

# Mutable aliasing example
a = [1, 2]
b = a    # b aliases the same list as a
b.append(3)
print(a)  # Output: [1, 2, 3]
print(a is b)  # True (same object)

# Immutable behavior
s1 = 'hi'
s2 = s1
s2 += '!'  # creates a new string and binds s2 to it
print(s1)  # 'hi'
print(s2)  # 'hi!'
print(s1 is s2)  # Usually False

# Shallow vs deep copy
import copy
orig = [[1], [2]]
sh = orig.copy()          # shallow copy
dp = copy.deepcopy(orig) # deep copy
sh[0].append(99)
print(orig)  # [[1, 99], [2]]  (inner list shared with shallow copy)
print(dp)    # [[1], [2]]    (deep copy unaffected)

What is Traversal?
Traversal is the process of visiting each element of a data structure (like a list, tuple, string, set, dictionary or a 2‑D list) so you can read or process the elements. Example uses: counting, searching, summing, filtering.

What is Iteration?
Iteration is the repeated execution of a block of code. In Python iteration is usually done with loops (for, while) or by using iterators and generator objects. When you iterate over an iterable (like a list), the loop visits elements one by one — this is traversal implemented by iteration.

Difference (brief):
Traversal is the goal (visit all elements). Iteration is the mechanism (repeating steps using loops or iterators).

Common iteration constructs in Python

• for loop: used to repeat a block for each element in an iterable.
• while loop: repeats while a condition holds; useful when number of steps is not known beforehand.
• Iterator protocol: objects implement __iter__() and __next__() so they can be used by for and next().
• Generator expressions and functions: produce values on the fly (memory efficient).

Useful built‑ins and idioms

• range(n) — sequence of integers 0..n-1, commonly used to get indices.
• enumerate(iterable) — gives (index, value) pairs while looping.
• zip(a, b) — iterate two sequences in parallel.
• items(), keys(), values() — iterate over dictionary entries.
• List comprehensions — concise way to build lists by iterating and optionally filtering.

Controlling iteration: break (exit loop), continue (skip to next iteration), and pass (do nothing).

Examples of traversal patterns

• Simple linear traversal: visiting all elements of a list.
• Indexed traversal: using indices to access or modify elements.
• Nested traversal: traversing a 2‑D list or matrix using nested loops.
• Conditional traversal: visit elements that meet a condition (filtering).

Iterator protocol (short)
An object is iterable if it implements __iter__() returning an iterator. An iterator implements __next__() which returns next value or raises StopIteration.

Good practices

• Prefer for loops over manual index loops when possible (safer, more Pythonic).
• Use enumerate() instead of maintaining a separate index counter.
• Avoid modifying a list while traversing it — iterate over a copy (list[:] ) if you must change the original.
• Use generators for large data to save memory.

Short code examples

# Linear traversal
numbers = [10, 20, 30]
for n in numbers:
    print(n)

# Using enumerate to get index and value
for i, val in enumerate(numbers):
    print(i, val)

# Traversing a matrix (2D list)
matrix = [[1,2,3],[4,5,6]]
for i in range(len(matrix)):
    for j in range(len(matrix[0])):
        print('element at', i, j, 'is', matrix[i][j])

# Using a generator (memory efficient)
def gen_squares(n):
    for i in range(n):
        yield i*i
for sq in gen_squares(5):
    print(sq)

When traversal and iteration matter
For tasks such as searching (linear scan), aggregation (sum, max, average), filtering (select elements that match a condition), transforming (map each element), and combining sequences (zip) — traversal + iteration are the core techniques.

Overview: Comprehensions and generator expressions are concise, readable ways to create new sequences or iterators from existing iterables using a compact syntax. They replace many simple for-loops and make transformations and filtering easy to express.

Types and basic syntax:

• List comprehension: [expression for item in iterable if condition]
• Set comprehension: {expression for item in iterable if condition}
• Dictionary comprehension: {key_expr: value_expr for item in iterable if condition}
• Generator expression: (expression for item in iterable if condition)

How they work:

- A comprehension evaluates the expression for each item in the input iterable and collects the results into a new container (list, set, dict). This builds the entire result in memory.

- A generator expression produces values on demand (lazy evaluation). It returns a generator object and yields items one by one as you iterate, so it uses far less memory for large streams.

Examples (short):

# List of squares
squares = [x*x for x in range(10)]

# Filtered list: even numbers
evens = [x for x in range(20) if x % 2 == 0]

# Dict: number -> square
sq_map = {x: x*x for x in range(6)}

# Set: unique word lengths from a sentence
words = 'this is a sample sentence'.split()
lengths = {len(w) for w in words}

# Generator: lazy squares for large range
gen = (x*x for x in range(10**7))  # no list created

Key behavioral notes:

• Comprehensions evaluate immediately and return a concrete container; generator expressions are lazy and return an iterator.
• Generator expressions can be passed directly to functions that consume iterables (e.g., sum(), any(), max()).
• Generators are exhausted after iteration; to reuse results you must recreate them or store the results.
• Comprehensions are generally as fast as equivalent for-loops (often faster in CPython) and usually clearer, but overly complex nested comprehensions hurt readability.

Nesting:

# Flatten a matrix
matrix = [[1,2,3],[4,5],[6]]
flat = [elem for row in matrix for elem in row]

# Nested comprehension with condition
pairs = [(i,j) for i in range(3) for j in range(3) if i != j]

Real-life use cases:

• Transforming CSV rows into dicts: [dict(zip(headers,row)) for row in rows]
• Filtering sensor readings: [r for r in readings if r.value > threshold]
• Streaming large log files and counting matches with a generator: sum(1 for line in open('log') if 'ERROR' in line)

Pitfalls & tips:

• Do not use huge list comprehensions when a generator can do the job (memory pressure).
• Prefer comprehensions for simple transformations; use functions or loops if logic is complex for readability.
• Remember that dict comprehensions require unique keys; later keys overwrite earlier ones.

Summary: Use list/set/dict comprehensions when you need a concrete container produced immediately. Use generator expressions when you want lazy evaluation, reduced memory usage, or to feed an iterable-consuming function.

Overview
Built‑in functions are pre-defined functions available in Python without importing anything (for example: len(), sum(), min()). Useful modules are collections of related functions and classes grouped in standard libraries that you import (for example: math, random, statistics, datetime).

Why they matter
They let you perform common tasks quickly and correctly: numerical operations, statistical summaries, date/time handling, random sampling, file & data handling. Using them avoids reinventing the wheel and makes code shorter and more reliable.

Common built‑in functions (short descriptions)

• len(obj) — length of sequence or collection.
• type(x) — type of an object.
• int(), float(), str() — type conversions.
• sum(iterable), min(), max(), round(), abs().
• sorted(iterable), reversed().
• map(func, iterable), filter(func, iterable), zip(), enumerate().
• any(), all() — boolean checks across iterables.
• divmod(a,b), pow(a,b).
• help(obj), dir(obj), isinstance(obj, Type) — introspection and debugging.

How to use modules
Import a module to access its functions. Example forms: import math, from math import sqrt, pi, import datetime as dt.

Key useful modules (class 11 focus)

• math — mathematical functions: sqrt(), factorial(), sin/cos, pi, etc.
• random — random numbers and choices: randint(), choice(), shuffle().
• statistics — mean, median, mode, variance, stdev: mean(), median().
• datetime — dates and times: datetime.now(), date arithmetic.
• json — read/write JSON data (json.load(), json.dump()).
• os, sys — file and system operations (optional for basic data tasks).

Small code examples

# statistics: mean, median
from statistics import mean, median
marks = [78, 82, 91, 67, 88]
print('Mean=', mean(marks))
print('Median=', median(marks))

# math: factorial and square root
import math
print('5! =', math.factorial(5))
print('sqrt(25)=', math.sqrt(25))

# random: simple sampling
import random
students = ['A', 'B', 'C', 'D']
print('Random pick:', random.choice(students))

# datetime: difference between dates
from datetime import date
d1 = date(2025, 1, 1)
d2 = date(2025, 10, 11)
print('Days between =', (d2 - d1).days)

# map and filter
nums = [1, 2, 3, 4]
sq = list(map(lambda x: x*x, nums))
even = list(filter(lambda x: x%2==0, nums))
print(sq, even)

Best practices

• Prefer standard library functions (they are tested and efficient).
• Import only what you need: from module import func or alias with as.
• Use help() and dir() to explore unfamiliar modules.

Class 11 focus
Understand how to call built‑ins, use map/filter/zip/enumerate with lists, and use the listed modules for solving common data problems: computing statistics, generating random samples, and handling dates.

CRUD stands for Create, Read, Update and Delete — the four basic operations you perform on data. In Python these operations are carried out on built-in data structures (lists, tuples, sets, dictionaries, strings) using specific methods or operators. Understanding how CRUD maps to each data structure helps you choose the right structure and write efficient code.

Create: make a new container or add new items. Examples: list append/extend, dict assignment, set add.

Read: access elements without changing them. Examples: indexing, slicing, iteration, dict.get, membership (in).

Update: change existing elements or replace containers. Examples: assignment to list index, list insert, dict[key] = value, set operations (union), note that tuples and strings are immutable so you create a new object instead of modifying in place.

Delete: remove elements or entire containers. Examples: del, pop, remove, clear, dict.pop, set.discard.

Mutability notes: Lists, dictionaries and sets are mutable (support in-place update/delete). Tuples and strings are immutable (you must create a new tuple/string when "updating").

Common pitfalls & good practices:

• Use dicts for key-based CRUD (fast average lookup/change).
• For ordered collections use lists; insertion/removal in middle is O(n).
• Check for membership with in before removing to avoid exceptions or use methods like discard for sets.
• Prefer dict.get(key, default) to avoid KeyError when reading.

Small code examples:

# List (mutable)
items = [10, 20]
# Create
items.append(30)
# Read
x = items[1]        # 20
# Update
items[0] = 15
# Delete
items.remove(20)

# Dictionary (mutable, key-based)
students = {'A001': 'Rita'}
# Create / Update
students['A002'] = 'Vikram'
students['A001'] = 'Rita Sharma'  # update
# Read
name = students.get('A001')
# Delete
students.pop('A002', None)

# Tuple (immutable)
t = (1, 2)
# To "update" create a new tuple
t = t + (3,)

# Set (mutable, unique elements)
s = {2, 3}
s.add(4)
if 3 in s:
    s.remove(3)

By mapping CRUD to appropriate Python methods and being aware of time/space costs, you can design simple data-driven programs (contact lists, inventories, student records) efficiently and safely.

What it is: Conversion between data structures in Python means transforming data from one built-in container type to another (list, tuple, set, dict, string) using constructors or methods so that the data can be used in an appropriate way (mutable/immutable, ordered/unordered, unique elements, key-value access).

Why it matters: Different tasks require different properties: lists for ordered mutable sequences, tuples for fixed records, sets for uniqueness and fast membership tests, and dictionaries for lookup by keys. Converting lets you use the best structure for each task.

Basic rules and behavior:

• list(iterable) — creates a list preserving order of the iterable.
• tuple(iterable) — creates an immutable tuple from the iterable.
• set(iterable) — creates an unordered collection with duplicates removed; order not guaranteed.
• dict(...) — constructs dictionaries. Passing a mapping or an iterable of key-value pairs (e.g., list of tuples) produces a dict. Passing just an iterable (like list of keys) does not by itself form key-value pairs.
• str.join(list_of_strings) and str.split(...) — convert between strings and lists of substrings.

Important consequences:

• Converting to set removes duplicates — use to get unique items (but original order is not preserved).
• Converting a dict to list returns a list of keys. Use dict.items() to get (key, value) pairs and dict.values() for values.
• Converting a list of two-item tuples to dict maps each first item to the second; duplicate keys get overwritten by the last occurrence.
• Mutability: converting to tuple makes data immutable; converting from tuple to list allows modification.

Complexities (brief): Most conversions are O(n) where n is the number of elements because each element must be visited.

Typical conversion functions and patterns:

# Examples of constructors and methods used for conversion
list(some_iterable)      # to list
tuple(some_iterable)     # to tuple
set(some_iterable)       # to set (unique elements)
str.join(list_of_strings)  # to string
some_string.split(sep)   # to list of substrings
dict(list_of_pairs)      # from [(key, value), ...]
list(a_dict)             # list of dict keys
list(a_dict.items())     # list of (key, value) tuples
list(a_dict.values())    # list of values
zip(keys, values)        # pairs two iterables; often wrapped with dict()

What are Nested Data Structures?

Nested data structures are data structures that contain other data structures as their elements. In Python this typically means lists, tuples, sets or dictionaries can hold other lists, tuples, sets or dictionaries. Nesting lets you represent complex, hierarchical data (tables, trees, JSON-like records) naturally.

Common nested types

• List of lists (2D lists): a matrix or table representation.
• List of dictionaries: a collection of records (e.g., list of student records).
• Dictionary of lists: grouped properties under keys (e.g., subjects -> marks list).
• Dictionary of dictionaries: hierarchical keyed data (e.g., JSON objects).
• Tuples inside lists or vice-versa: fixed-size records inside a mutable collection.

How to access elements

Accessing nested elements is done by chaining index or key operations. Examples:

# 2D list: matrix[i][j]
matrix = [[1,2],[3,4]]
val = matrix[0][1]  # 2

# nested dict: d['a']['b']
d = {'a': {'b': 10}}
val = d['a']['b']  # 10

# list of dicts: students[0]['name']
students = [{'name': 'Alice', 'age': 15}, {'name': 'Bob', 'age': 16}]
name0 = students[0]['name']  # 'Alice'

Iteration and nested loops

To process nested structures, use nested loops or nested comprehensions. For example, iterating a 2D list uses one loop per dimension:

for row in matrix:
    for item in row:
        print(item)

# Nested comprehension to flatten a 2D list
flat = [item for row in matrix for item in row]

Modification and mutability

Be aware of mutability: lists and dictionaries are mutable; tuples are not. If you store a mutable object (like a list) inside multiple places, modifying it in one place affects all references. Use copy.copy or copy.deepcopy when needed.

Common operations and cautions

• Flattening: convert nested lists into a single list (via nested loops or comprehensions).
• Depth (nesting level): keep track to avoid deep recursion; Python has recursion limits.
• Shallow vs deep copy: shallow copy copies references to inner objects; deep copy duplicates nested contents.
• Performance: nested loops increase time complexity multiplicatively (see formulas below).

Real-life use-cases

• Spreadsheet data: 2D lists for rows and columns.
• JSON APIs: dictionaries with nested dictionaries and lists for structured responses (users → posts → comments).
• School records: list of dicts where each dict stores name, grades (list), and address (dict).
• Graphs and trees: adjacency lists (dict of lists) or nested dicts for nodes and properties.

What it is: Error handling related to data operations means writing Python code that anticipates, detects and responds to errors that occur when reading, writing, converting, or validating data (files, user input, CSV/JSON, lists/dicts, etc.). Good error handling keeps programs robust, prevents crashes, and makes failures informative.

Why it matters: Data is often messy or unavailable: files may be missing, numbers may be malformed, fields may be missing, or external resources may be temporarily unreachable. Without handling these situations a program will raise exceptions and stop. Proper handling lets you give defaults, retry, log problems, or fail gracefully.

Common exception types in data operations

• FileNotFoundError / IOError: file missing or unreadable.
• ValueError: converting a string to int/float fails (e.g., int("abc")).
• TypeError: operation applied to wrong type (e.g., adding str and int).
• IndexError: accessing invalid list index.
• KeyError: missing key in a dictionary.
• ZeroDivisionError: division by zero when computing statistics.
• json.JSONDecodeError / UnicodeDecodeError: malformed JSON or wrong file encoding.

Techniques and patterns

• Use try/except to catch specific exceptions and handle them. Prefer catching specific exception classes rather than a bare except:.
• Use else to run code when no exception occurred, and finally to always run cleanup (close resources) if not using with.
• Use with open(...) to ensure files are closed automatically.
• Validate and sanitize input before converting: strip whitespace, check numeric pattern, or use helper functions that return defaults on failure.
• Raise meaningful exceptions with raise ValueError('message') when detecting invalid data inside functions (fail fast).
• Log errors (using the logging module) so the cause can be diagnosed later rather than silently ignoring problems.
• Provide fallbacks or retry logic for transient errors (e.g., network read). Keep retries limited and use backoff if needed.

Best practices

• Catch the narrowest exception you expect.
• Keep try blocks small — only include the statement(s) that might raise the expected error.
• Don’t use exceptions for normal control flow.
• Provide clear error messages and use logging for diagnostics.
• Validate input and data formats before processing.

Short code patterns

# safe file open with fallback
try:
    with open('data.csv') as f:
        text = f.read()
except FileNotFoundError:
    print('data.csv not found — using default data')
    text = ''

# safe conversion with default
try:
    x = int(user_input)
except ValueError:
    x = 0  # fallback or ask again

# raise when encountering invalid record
def parse_age(s):
    try:
        age = int(s)
    except ValueError:
        raise ValueError(f'Invalid age: {s}')
    if age < 0:
        raise ValueError('Age must be >= 0')
    return age

In sum: anticipate likely problems from external or untrusted data, catch and handle specific exceptions, validate input early, and ensure resources are always cleaned up.

'Practical Examples and Problem Solving' in the 'Working with Data in Python' chapter teaches how to apply Python to collect, clean, analyze and present data to solve real-world problems. The process follows these steps:

• Understand the problem: identify inputs, outputs and constraints.
• Choose data structures: lists, tuples, dictionaries, sets for different needs (ordered data, key-value lookup, unique elements).
• Data ingestion: read data from user input, text files or CSV (use the csv module or pandas for larger tasks).
• Data cleaning & validation: handle missing values, convert types (strings to int/float), strip whitespace.
• Processing & algorithms: counting, searching, filtering, sorting, aggregations (sum, mean, median), grouping by keys.
• Output & visualization: textual reports, tables or graphs (bar/histogram/line/scatter) to communicate results.

Key Python constructs to use:

• Loops and conditionals (for, while, if)
• Comprehensions (list/dict/set) for concise transformations
• Functions to modularize tasks
• File handling (open/read/write) and the csv module for structured data
• Dictionaries to implement frequency counts and grouping

Small example workflow (student marks analysis): read marks from a CSV, convert to numbers, compute mean/median/mode, find top scorers, and plot distribution.

# Example: read marks, compute mean
import csv
marks = []
with open('marks.csv') as f:
    reader = csv.reader(f)
    next(reader)  # skip header
    for row in reader:
        marks.append(int(row[1]))

mean = sum(marks)/len(marks)

Problem solving tips:

• Start with small test data and print intermediate results.
• Break the problem into functions (read_data, clean_data, analyze, visualize).
• Use dictionaries for counting (e.g., frequency of grades) and sorting with key functions.
• Handle edge cases (empty files, non-numeric entries) with try/except or validation checks.