Chapter 3 — File Handling

What is a file? A file is a named collection of related information stored permanently on a secondary storage device (disc, SSD). Files let programs store data between runs. In programming, a file is accessed through a file object/handle that the program uses to read from or write to the file.

Why file handling? Programs often need to preserve data (user settings, logs, records, datasets). File handling provides standard operations to create, read, update and delete persistent data.

Basic concepts

• File types: text files (human-readable, e.g. .txt, .csv) and binary files (non-text, e.g. images, serialized objects).
• File modes: Common modes in Python-style notation: 'r' (read), 'w' (write, truncate), 'a' (append), 'r+' (read/write), and binary variants like 'rb', 'wb'.
• File pointer (cursor): indicates the current byte position in the file. Operations like read() and write() move the pointer. You can query position with tell() and change it with seek().
• Common operations/methods: open(), read(), readline(), readlines(), write(), writelines(), close(), seek(), tell(). Also context manager with ensures automatic close.

Typical workflow

1. Open the file (specify mode).
2. Perform read/write operations.
3. Close the file (or use with to auto-close).

Small Python examples (illustrative)

# Write text to a file
with open('students.txt', 'w') as f:
    f.write('101,Ajay,87\n')
    f.write('102,Sima,92\n')

# Read whole file
with open('students.txt', 'r') as f:
    data = f.read()

# Read line by line
with open('students.txt', 'r') as f:
    for line in f:
        roll, name, marks = line.strip().split(',')
        print(roll, name, marks)

# Append a log entry
from datetime import datetime
with open('app.log', 'a') as log:
    log.write(f'{datetime.now()}: User logged in\n')

# Binary example: storing an object using pickle
import pickle
student = {'roll':103, 'name':'Rahul', 'marks':90}
with open('student.obj', 'wb') as bf:
    pickle.dump(student, bf)

Good practices

• Use with to auto-close files and avoid resource leaks.
• Open files in the smallest necessary mode (read-only if only reading).
• Handle exceptions (e.g., file not found, permission errors).
• Prefer CSV/JSON for structured text data to ease parsing and interoperability.

When to use fixed-length records vs. variable-length

Fixed-length records (each record same byte size) make random access and offset calculation easy; variable-length records are more space-efficient for differing field sizes but require indexes or delimiters.

What is a file type/format?
A file type or format defines how data is organized and stored in a file so programs can read, interpret, and write it. It includes conventions for encoding, metadata (header), record layout, and sometimes compression.

Broad classification

• Text files: Human-readable characters encoded with a charset (ASCII, UTF-8). Example: .txt, .csv, .html.
• Binary files: Encoded in bytes not directly readable. Example: executables, compiled objects, images, audio, video.
• Structured files: Contain records and fields with a known structure. Example: CSV, JSON, XML. Often used for data interchange and storage.
• Multimedia files: Specialized binary formats for images, audio, video. Example: .jpg, .png, .mp3, .wav, .mp4.
• Compressed and archive formats: Store one or more files with compression. Example: .zip, .tar.gz, .rar.
• Database/Indexed files: Support random access via indexes. Example: .db, .sqlite.

Key components of many file formats

• Header: Metadata (magic number, version, sizes, encoding) at file start.
• Body (records): Actual data; can be fixed-length or variable-length records.
• Footer/Trailer: Optional checksums, indexes, or end markers.

Record types and access

• Fixed-length records: Each record occupies same number of bytes — makes random access easy (compute offset directly).
• Variable-length records: Use delimiters or length fields; need scanning or an index for direct access.
• Sequential access: Read from start toward end (typical for text and many binary streams).
• Random access: Jump to byte offsets using file pointer (typical for fixed-length records, indexed files).

Encoding and compatibility

• Character encoding: ASCII, UTF-8, UTF-16; mismatch causes garbled text.
• Endianness: Byte order for multi-byte binary values (big-endian vs little-endian) matters in binary formats.
• File extension vs MIME type: Extension helps OS/apps choose the handler; MIME type used in web/email context.

Real-life considerations

• Choose CSV or JSON for tabular/structured data interchange.
• Use UTF-8 for text to maximize compatibility.
• Use appropriate formats for media: PNG for lossless images, JPEG for photos (lossy), MP3/AAC for compressed audio.
• Compress large files before transfer to save bandwidth (ZIP, gzip).
• Design file headers and indexes if you need fast random access on large datasets.

Short example: fixed-length record layout

Header (100 bytes)
Record 0 (100 bytes)
Record 1 (100 bytes)
...

To access record i: seek to byte offset = header_size + i * record_length.

What it means
Opening a file establishes a connection between a program and a file stored on disk so the program can read from or write to it. Closing a file ends that connection and frees system resources. Properly opening and closing files ensures data integrity (buffers are flushed), avoids resource leaks, and prevents file corruption.

Key concepts

• File object/descriptor: The handle returned by the operating system or language runtime when a file is opened. It is used for further operations (read, write, seek).
• Modes: Define how the file is accessed (read, write, append, binary/text, read+write). Common modes: 'r', 'w', 'a', 'r+', 'w+', 'a+', and their binary forms 'rb', 'wb', etc.
• Buffering and flush: Writes are usually buffered for efficiency. Closing or flushing forces buffered data to be physically written to disk.
• File pointer (cursor): The current byte/character position inside the file used for subsequent read/write. It can be moved using seek() and located using tell().
• Errors: Trying to open a non-existent file in write modes creates or overwrites it ('w'), while opening in read mode ('r') raises an error if it doesn't exist.

Common steps when working with files

1. Open the file in required mode (request a file object).
2. Perform required operations (read, write, iterate, seek).
3. Flush (if necessary) and close the file to release resources and ensure all data is written.

Best practice
Use a context manager (or language-specific equivalent) that automatically closes the file, e.g., in Python: with open(...):. If not using a context manager, ensure close() is called in a finally block to guarantee closure even on exceptions.

Error handling
Always handle exceptions produced while opening or operating on files (e.g., FileNotFoundError, PermissionError) so that files are not left open and the program can respond gracefully.

Text vs Binary
Text mode handles character encoding and newline translation. Binary mode works with raw bytes (useful for images, executables, compressed files).

What are File Modes?

File modes determine how a program opens a file — whether for reading, writing, appending, or a combination — and what happens if the file does or does not exist. In Python (and many languages) modes are expressed as short strings such as 'r', 'w', 'a', optionally combined with '+' (read/write) or 'b' (binary) or 't' (text).

Common modes and their behavior:

• r — Open for reading. File must exist. File pointer at start (offset 0).
• w — Open for writing. Creates file if it does not exist; if it exists, truncates (empties) it. Pointer at start.
• a — Open for appending. Creates file if it does not exist. Writes always go to end-of-file (EOF); pointer for writing is at EOF.
• x — Create exclusively: fail if file already exists. Useful to avoid accidental overwrite.
• r+ — Open for both reading and writing. File must exist. Pointer at start; you can read and write (writing may overwrite existing bytes unless you seek).
• w+ — Open for reading and writing. Creates if needed; truncates existing file to zero length. Pointer at start.
• a+ — Open for reading and writing. Creates if needed. Writes are always appended; reads can occur but writing moves to EOF.
• Binary vs Text: Add 'b' to open in binary mode (e.g. 'rb', 'wb', 'ab') for non-text files such as images or executables. Text mode (default, 't') handles encoding and newline conversions.

Key practical points:

• When a mode truncates the file (e.g., 'w', 'w+'), existing data is lost immediately on open.
• Append modes ('a', 'a+') ensure that writes do not overwrite existing data — they go to EOF.
• In modes allowing both read and write, the file pointer controls where reading/writing occurs; use seek() to move the pointer as needed. In append modes, writes usually ignore seek and go to EOF.
• Use with open(filename, mode) as f: to automatically close files and avoid resource leaks.

Example behavior summary (short):

• r: read only, must exist, pointer=0.
• w: write only, creates or truncates, pointer=0.
• a: append only, creates if missing, writes at EOF.
• r+/w+/a+: read/write combinations with behaviors above.

Best practices: Use 'r' when only reading; use 'a' for logs; use 'x' to avoid overwriting; prefer 'rb'/'wb' for non-text data; always handle exceptions (FileNotFoundError, PermissionError) and close files or use context managers.

What is reading from files?

Reading from files is the process of retrieving stored data from a file on disk into a program's memory so the program can process it. In Class 12 Computer Science (Python), reading is done using file objects returned by the open() function and methods such as read(), readline(), and readlines(), or by iterating directly over the file object.

Key concepts

• Modes: common modes for reading are r (text read), rb (binary read), and r+ (read/update). Using with open(...) is recommended because it auto-closes the file.
• Cursor (file pointer): every file object has a pointer indicating the next byte/character to read. Methods advance the pointer; seek() moves it and tell() reports its position.
• Text vs Binary: text mode returns str (decoded using an encoding like UTF-8); binary mode returns bytes.
• Efficiency: read() loads specified bytes/chars (or whole file if no arg). For large files, read in chunks or iterate line-by-line to avoid memory issues.
• Structured files: CSV, JSON, logs need parsing after reading (use csv or json modules in Python for convenience).

Common reading patterns

• Read entire file into a single string: text = f.read()
• Read a fixed number of characters/bytes: chunk = f.read(n)
• Read one line at a time: line = f.readline() or loop: for line in f:
• Get all lines as a list: lines = f.readlines() (careful: uses memory proportional to number of lines)
• Move pointer and re-read: f.seek(0) to go to start

Error handling and encodings

When reading files, handle exceptions such as FileNotFoundError or UnicodeDecodeError. Specify encoding when opening text files (e.g., open('file.txt', 'r', encoding='utf-8')).

Best practices

• Use with open(...) so files are closed automatically.
• For large files read in chunks: while True: chunk = f.read(1024); if not chunk: break.
• Prefer iterating over the file for line-by-line processing: for line in f:
• For CSV/JSON use the csv and json modules rather than manual parsing.

Small conceptual example (explain flow)

Open file -> file object created -> read data (pointer advances) -> process data in memory -> close file (or auto-close via with).

What it means
Writing to files means sending data from a program to a persistent storage file so it can be read later. In Python (CBSE Class 12), this is done by opening a file in a write-related mode and using methods like write() or writelines(). Files can be text (human readable) or binary (bytes).

Common modes
'w' — write (creates or truncates), 'a' — append (creates if not exists), 'x' — create exclusively (fail if exists), 'r+' — read and write without truncation. Add 'b' for binary and 't' for text (default).

Basic pattern
1) Open the file with the required mode. 2) Write data using methods. 3) Flush (optional) and close the file. Best practice: use a context manager (with) so file is closed automatically.

Important methods
open(filename, mode, encoding=None), file.write(string), file.writelines(iterable_of_strings), file.flush(), file.close(), file.seek(offset, whence), file.tell().

Example (text file, overwrite):

with open('notes.txt', 'w', encoding='utf-8') as f:
    f.write('This is a line.\n')
    f.writelines(['Second line.\n', 'Third line.\n'])

Binary write example:

with open('image_copy.jpg', 'wb') as f:
    f.write(binary_data)

Key considerations
- 'w' truncates the file immediately; use 'a' to preserve existing contents.
- Use correct encoding (e.g., 'utf-8') for text.
- Handle exceptions (IOError/OSError) when disk full or permission denied.
- Use seek() and tell() when you need random access.
- For large writes, consider buffering or writing in chunks to avoid memory issues.

Error handling pattern

try:
    with open('data.csv', 'a', encoding='utf-8') as f:
        f.write(new_line)
except OSError as e:
    print('Write failed:', e)

Overview
A file pointer is an internal cursor that indicates the current byte position in an open file. Sequential file operations (read/write) move the pointer forward as data is processed. Random access is the ability to move the file pointer to any byte position and then read or write there, allowing non-linear (direct) access to file contents.

Why it matters
Sequential access requires reading through earlier data to reach a point; random access lets you jump directly to the needed part. Random access is essential for efficient updates of fixed-length records, multimedia seeking, databases, and resume operations.

Key concepts and operations

• tell()/ftell()/getFilePointer() — return the current byte offset (position) of the file pointer from the file start.
• seek()/fseek()/RandomAccessFile.seek() — move the file pointer to a specified position. Usually accepts an offset and a "whence" (reference) value.
• Whence/reference values — common values: SEEK_SET (beginning of file), SEEK_CUR (current position), SEEK_END (end of file).
• Binary vs text mode — for predictable byte offsets and random-access calculations, use binary mode; text mode may do newline translations on some platforms which affect offsets.
• Fixed-length records — simplest for random access: each record occupies the same number of bytes so you can compute offsets directly.

Common API examples

• C (std I/O): fseek(fp, offset, whence), ftell(fp). Whence: SEEK_SET, SEEK_CUR, SEEK_END.
• Python: file.seek(offset, whence=0), file.tell(). Whence values: 0 (start), 1 (current), 2 (end).
• Java: RandomAccessFile raf = new RandomAccessFile("file", "rw"); raf.seek(pos); raf.getFilePointer();

Example usage (conceptual)

// C-like pseudocode: jump to record 5 (0-based) when each record is 100 bytes
int recordIndex = 5;             // 0-based index
int recordSize = 100;            // bytes per record
long offset = recordIndex * recordSize; // compute byte offset
fseek(fp, offset, SEEK_SET);     // move file pointer
fread(buffer, recordSize, 1, fp);

When random access is harder
If records are variable-length (e.g., text lines of differing sizes), you cannot compute record offsets without an index or scanning. Typical solutions: maintain an index table (list of offsets) or use a database format that stores offsets.

Advantages

• Fast direct updates of specific records.
• Efficient seeking in large media files (audio/video).
• Useful for implementing file-based indices and simple databases.

Limitations & cautions

• Random writes can corrupt file structure if record sizes change; prefer fixed-size records or rewrite following data.
• Text mode may make offsets unreliable—use binary mode for byte-accurate operations.
• Concurrent access requires locking to avoid race conditions when multiple processes change the same region.

What it is
A context manager in Python is an object that defines a runtime context for resource management (open files, locks, DB connections, temporary directories, etc.). The with statement uses the context manager protocol to ensure setup and guaranteed cleanup, even if exceptions occur.

Why use it
It simplifies code that acquires and releases resources by replacing try/finally blocks and reducing the risk of forgetting to release the resource.

Protocol (how it works)
A context manager implements two methods:
def __enter__(self): — acquire resource, optionally return it to the with-block
def __exit__(self, exc_type, exc_value, traceback): — release the resource; if it returns True, it suppresses the exception.

Semantic expansion
The statement with EXPR as VAR: BLOCK is conceptually equivalent to:

mgr = (EXPR)
enter = type(mgr).__enter__
exit = type(mgr).__exit__
value = enter(mgr)
try:
    VAR = value
    BLOCK
finally:
    exit(mgr, *sys.exc_info())

Key behaviors
- __enter__ is called before the block runs. Its return value (if any) is bound to the name after as.
- __exit__ is always called after the block ends, even if the block raised an exception.
- If __exit__ returns True, the exception (if any) is suppressed; otherwise it propagates.

Built-in and convenient uses
Common built-in context manager: open() for files. The contextlib module helps create custom context managers (class-based or generator-based with @contextmanager).

Short examples (in HTML-safe code)

# File handling (recommended)
with open('data.txt', 'r') as f:
    text = f.read()
# file is automatically closed here

# Custom class-based context manager
class MyCM:
    def __enter__(self):
        print('acquire')
        return 'resource'
    def __exit__(self, exc_type, exc_value, tb):
        print('release')
        # return False (or None) to propagate exceptions

with MyCM() as r:
    print(r)

# Generator-based context manager using contextlib
from contextlib import contextmanager
@contextmanager
def my_cm():
    print('setup')
    try:
        yield 'res'
    finally:
        print('cleanup')

with my_cm() as r:
    print(r)

When to use
Use context managers whenever you need deterministic setup/cleanup: files, network sockets, locks, transactions, temporary directories, monkey-patched state, profiling/timing and more.

Overview: File operations (open, read, write, close, delete) can fail for many reasons: missing files, permission problems, disk full, corrupt data, wrong mode, encoding errors, etc. Such failures raise exceptions which, if not handled, crash the program. Error and exception handling ensures programs respond safely, release resources, and give meaningful messages or recovery actions.

Error vs Exception: An error is a problem (e.g., hardware/disk) while an exception is a language-level object representing a runtime problem (e.g., FileNotFoundError). Exceptions can be caught and handled.

Common file-related exceptions (Python names): FileNotFoundError, PermissionError, IsADirectoryError, EOFError, IOError/OSError, UnicodeDecodeError/UnicodeEncodeError, ValueError.

Basic handling pattern: Use try/except to catch expected exceptions, optionally use else for code that should run when no exception occurred, and finally for cleanup. Better: use the context manager (with) to auto-close files.

# Example: safe read with handling
try:
    f = open('config.txt', 'r', encoding='utf-8')
    data = f.read()
except FileNotFoundError:
    print('Config missing — using defaults')
except PermissionError:
    print('Cannot read config — check permissions')
except UnicodeDecodeError:
    print('Encoding error in file')
else:
    process(data)
finally:
    try:
        f.close()
    except NameError:
        pass

Preferred approach — context manager:

# Auto-closes file and is concise
try:
    with open('data.csv', 'r', encoding='utf-8') as f:
        for line in f:
            process_line(line)
except FileNotFoundError:
    handle_missing()
except PermissionError:
    handle_perm()

Checking vs EAFP: Two common styles:

• Look Before You Leap (LBYL): check conditions before operations, e.g., if os.path.exists(file): open(...)
• Easier to Ask Forgiveness than Permission (EAFP): try the operation and catch exceptions. EAFP is idiomatic in Python and avoids race conditions.

Raising and re-raising exceptions: You can raise exceptions to signal problems or re-raise to let higher-level code handle them.

if not valid_format(data):
    raise ValueError('Invalid file format')

try:
    do_work()
except Exception as e:
    log(e)
    raise   # re-raise after logging

Custom exceptions: For clearer program structure define custom exception classes (inherit from Exception) to represent domain-specific file errors.

class ConfigError(Exception):
    pass

if config_missing:
    raise ConfigError('Mandatory config entry X missing')

Best practices:

• Prefer with to open files: it guarantees close even on errors.
• Catch specific exceptions, not a bare except, to avoid hiding bugs.
• Clean up resources (temporary files, locks) in finally blocks or use context managers.
• Validate inputs and use safe write patterns (write to temp file then atomically rename) to avoid data corruption.
• Log exceptions with enough context to debug (file name, operation, user id) but avoid leaking secrets.
• Use encoding arguments when opening text files to avoid Unicode errors.

What is a CSV file?
CSV (Comma-Separated Values) is a plain-text format for tabular data where each line is a record and fields are separated by a delimiter (commonly a comma). The first line often contains a header with column names.

Structure

• Records: one per line
• Fields: separated by a delimiter (comma, semicolon, tab)
• Headers: optional first row with column names
• Quoting/escaping: fields that contain delimiters, newlines or quotes are enclosed in quotes and quotes inside are escaped (e.g., "" for a quote in RFC 4180).

Why use CSV? It is simple, human-readable, widely supported (spreadsheet apps, databases, programming languages) and good for exchanging tabular data.

Limitations

• No strict schema (types not enforced)
• Problems with embedded delimiters, newlines, or encodings if not handled properly
• Large files may need streaming (not loading whole file into memory)

Common operations

• Open and read: line-by-line parsing or using CSV libraries
• Write: create header row, then write records with correct quoting
• Append: open file in append mode and write new rows
• Convert: to/from Excel, JSON, databases
• Filter/aggregate: compute sums, averages, counts, group-by operations

Best practices

• Use a CSV library (not plain split) to handle quoting and edge cases
• Specify encoding (UTF-8) and newline handling when opening files
• Validate header and expected column count
• Stream large files (process rows one at a time) to save memory

Python examples (conceptual)

# Reading using csv module
import csv
with open('students.csv', 'r', encoding='utf-8', newline='') as f:
    reader = csv.DictReader(f)  # uses header row
    for row in reader:
        print(row['Name'], row['Marks'])

# Writing using csv module
with open('out.csv', 'w', encoding='utf-8', newline='') as f:
    writer = csv.writer(f)
    writer.writerow(['Name','Marks'])
    writer.writerow(['Asha', 85])

Error handling

• Inconsistent column counts: skip or pad rows
• Missing values: treat as empty or fill default values
• Type conversion: validate and convert strings to int/float/date with try/except

Memory and performance
Reading line-by-line (streaming) is O(n) time and O(1) extra memory; loading entire file into a list is O(n) memory. Use buffered I/O and libraries optimized for CSV when available (pandas for large numeric data analytics).

What is JSON? JSON (JavaScript Object Notation) is a lightweight text format for exchanging structured data. It is human-readable and language-independent. JSON files typically use the extension .json.

Basic structure:

• Object: a collection of key/value pairs enclosed in braces. Example: {"name": "Asha", "age": 17}
• Array: an ordered list of values enclosed in brackets. Example: [1, 2, 3]
• Values: string, number, object, array, true, false, null

Why use JSON in file handling? JSON is commonly used to store structured data (settings, records) and to exchange data between client and server (APIs). In file handling, we read JSON from files, convert it into program data structures (deserialization), modify it, and write it back (serialization).

Working with JSON in Python (typical CBSE approach)

The Python json module provides functions to convert between JSON text and Python objects:

• json.load(file) — read JSON from an open file and parse into Python objects (dict, list, etc.)
• json.loads(string) — parse JSON from a string
• json.dump(obj, file) — serialize Python obj to JSON and write to an open file
• json.dumps(obj) — return JSON string from Python obj

Typical read-modify-write pattern:

with open('data.json', 'r') as f:
    data = json.load(f)        # read and parse

# modify Python object 'data'

with open('data.json', 'w') as f:
    json.dump(data, f, indent=4)  # write nicely formatted JSON

Important notes:

• JSON keys must be strings (in double quotes in the JSON text).
• When writing, use indent for readability; without it JSON is compact (no extra whitespace).
• Be careful with file modes: use 'r' for reading, 'w' for overwriting, 'a' for appending (appending JSON may require reading, updating, then rewriting to keep valid JSON).
• Handle exceptions: json.JSONDecodeError when parsing invalid JSON; IOError when file operations fail.

Memory and large files: For very large JSON files, avoid loading the entire file into memory. Use streaming parsers or process the file in chunks (or store data in a database).

Overview

Binary files store data in a compact, non-human-readable binary form (bytes). Serialization is the process of converting in-memory objects into a byte stream so they can be written to a binary file, sent over a network, or stored for later reconstruction (deserialization).

Binary files vs Text files

• Text files: data is stored as readable characters (ASCII/Unicode). Easier to inspect and edit, but larger for complex structures.
• Binary files: data stored in machine-friendly binary representation. More compact, faster to read/write, preserves exact types and object structure.

Why serialization?

• Persist complex objects (objects with fields referencing other objects) without manual conversion to text.
• Transmit objects over network (RPC, sockets, RMI).
• Cache objects to disk and restore later with original types preserved.

Java (Class 12 CBSE context) — how serialization works (basic)

In Java, an object is serializable if its class implements java.io.Serializable. The ObjectOutputStream converts the object into bytes and writes to an OutputStream (e.g., FileOutputStream). ObjectInputStream reads bytes and reconstructs the object.

import java.io.*;

// example class
class Student implements Serializable {
  private static final long serialVersionUID = 1L; // recommended
  String name;
  int age;
  transient String password; // not serialized
}

// Writing (serialization)
ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("students.bin"));
out.writeObject(studentInstance);
out.close();

// Reading (deserialization)
ObjectInputStream in = new ObjectInputStream(new FileInputStream("students.bin"));
Student s = (Student) in.readObject();
in.close();

Key Java concepts

• Serializable: marker interface; no methods but marks class for default serialization.
• serialVersionUID: a long value that identifies class version. If not declared, JVM computes one. If class structure changes and serialVersionUID differs, deserialization fails with InvalidClassException. Recommended to declare: private static final long serialVersionUID = 1L;
• transient: fields marked transient are not serialized (e.g., passwords, cached values).
• static fields are not serialized (belong to class, not instance).
• Custom serialization: define private methods writeObject(ObjectOutputStream) and readObject(ObjectInputStream) to control how fields are written/read.
• Externalizable: an interface (extends Serializable) that forces the class to implement writeExternal/readExternal to control full serialization process.

Advantages and limitations

• Advantages: preserves object graphs, compact storage, faster I/O, easy to persist and transmit objects.
• Limitations: not human-readable, fragile to class changes unless serialVersionUID managed, potential security risks (deserializing untrusted data), language-specific formats (Java serialization not directly readable by other languages).

Best practices

• Always declare serialVersionUID for classes you serialize.
• Use transient for sensitive information or values you can recompute.
• Validate data in readObject() after deserialization.
• Avoid Java serialization for long-term storage or cross-language persistence; consider standardized formats (JSON, Protocol Buffers) for interoperability.

Security note

Never deserialize data from untrusted sources without validation — deserialization can be exploited to run malicious code. Consider object-input validation or safer alternatives.

Overview
In Python, the standard library modules os and shutil provide tools to work with files and directories. os gives low-level utilities (paths, current directory, listing, creation, removal, metadata) and shutil provides higher-level file and directory operations (copy, move, remove tree, archive).

Key concepts

• Paths: Represent filesystem locations. Use os.path helpers to build and inspect paths portably (os.path.join, os.path.abspath, os.path.exists, os.path.isdir, os.path.isfile).
• Current working directory: Use os.getcwd() to get and os.chdir(path) to change it.
• Create/delete directories: os.mkdir(), os.makedirs() (recursive), os.rmdir(), os.removedirs() and shutil.rmtree() (remove non-empty tree).
• List and traverse directories: os.listdir(), os.scandir(), and os.walk() for recursive traversal.
• File operations: os.remove()/os.unlink() to delete single files, os.rename() to rename or move, shutil.copy()/shutil.copy2()/shutil.copyfile() to copy files, and shutil.move() to move files or directories.
• Metadata & permissions: os.stat(path) gives size, modification time etc.; os.chmod() changes permissions (platform-dependent).

Safety and best practices

• Always check existence (os.path.exists()) before reading, writing, or deleting. Use try/except to catch OSError or FileNotFoundError.
• Use os.path.join() instead of string concatenation to build paths to remain platform-independent.
• When deleting directories that may contain files, prefer shutil.rmtree() with caution and perhaps confirm with user input.
• Use with open(...) context manager when working with file objects to ensure files are closed properly.

Small code patterns

# create a directory if it doesn't exist
if not os.path.exists(folder_path):
    os.makedirs(folder_path)

# copy file preserving metadata
shutil.copy2(src_file, dst_file)

# iterate files recursively
for dirpath, dirnames, filenames in os.walk(root):
    for f in filenames:
        process(os.path.join(dirpath, f))

Real-life uses: organizing downloaded files into folders by extension, backing up user data (copy trees to backup location), cleaning temporary directories, deploying application files (move/rename), counting & summarizing files in a project, packaging (shutil.make_archive).

Overview
File handling is the set of operations used to create, open, read, write, update and delete files stored on disk. In Class 12 Computer Science practicals you implement real-world tasks — storing student records, logs, reports, CSV data, binary objects — using text and binary files. Key concepts: file modes (read, write, append), sequential vs random access, read/write functions, file pointers (seek/tell), and safe handling (using exceptions and the with context manager).

Basic operations
1) Open a file: choose mode—'r' (read), 'w' (write, truncates), 'a' (append), 'rb'/'wb' (binary modes).
2) Read: read(), readline(), readlines().
3) Write: write(), writelines().
4) Close: ensure resources freed (use with to auto-close).
5) Random access: seek(offset, whence) and tell() to move/read from specific byte positions in binary or text files.

File types
Text files store human-readable characters (plain .txt, .csv). Binary files store bytes (images, serialized objects, fixed-length records).

Common patterns in practical programs
- CRUD (Create, Read, Update, Delete) for records: sequential scanning with a temporary file for updates/deletes.
- Parsing CSV files to store and compute (marks processing, totals, averages).
- Searching and replacing text (simple string operations or regex).
- Copy/backup files (read chunks and write to new file).
- Fixed-length record files for direct access using computed offsets.

Best practices
- Use with open(...) to avoid forgetting to close files.
- Handle exceptions (file not found, permission errors).
- When updating, write to a temporary file then replace original to avoid data loss.
- For large files, read in chunks/stream rather than loading whole file into memory.

Real-life uses
Payroll and student databases (flat-file storage), server logs processing, exporting/importing CSV spreadsheets, backups and file copy utilities, storing program settings and simple caches.

Small code examples (Python style)

# 1. Create and write to a file
with open('students.txt', 'w') as f:
    f.write('101,Anita,85\n')
    f.write('102,Raj,78\n')

# 2. Read and count lines/words/characters
with open('students.txt', 'r') as f:
    text = f.read()
lines = text.splitlines()
num_lines = len(lines)
num_words = len(text.split())
num_chars = len(text)

# 3. Copy a file in chunks (binary-safe)
with open('image.jpg', 'rb') as src, open('copy.jpg', 'wb') as dst:
    while True:
        chunk = src.read(4096)
        if not chunk:
            break
        dst.write(chunk)

# 4. Update a record using a temp file
import os
with open('students.txt', 'r') as src, open('tmp.txt', 'w') as tmp:
    for line in src:
        sid, name, marks = line.strip().split(',')
        if sid == '102':
            marks = '82'  # update
        tmp.write(','.join([sid, name, marks]) + '\n')
os.replace('tmp.txt', 'students.txt')

# 5. Random access (fixed-length records)
# Suppose each record is exactly 32 bytes; to read record i:
record_size = 32
i = 5  # 0-based
with open('records.bin', 'rb') as f:
    f.seek(i * record_size)
    record = f.read(record_size)

Overview
File handling is more than open-read-write-close. Best practices ensure data integrity, availability and confidentiality, while security measures protect files from unauthorized access, tampering and data loss. Combining good coding patterns with system-level controls prevents bugs, exploits and privacy breaches.

Core principles

• Principle of Least Privilege: grant the minimum file access rights needed (read, write, execute) to users and processes.
• Fail-safe defaults: deny access by default and allow only approved actions.
• Defense in depth: combine application-level checks, OS permissions, encryption and network security.
• Auditability: keep logs to trace who accessed or modified files.
• Data integrity and availability: use checksums, hashes, backups and atomic operations.

Practical best practices (programming and system)

• Always close files (or use context managers). Example: in Python use with open(...) as f: to auto-close.
• Handle exceptions around file operations to avoid resource leaks and partial writes (try/catch/finally or with-statement).
• Use atomic writes for critical data: write to a temporary file, fsync, then rename/move to final name so readers never see partial content.
• Sanitize and validate file paths and filenames: avoid path traversal ("../") and remove dangerous characters.
• Validate uploaded files: check size limits, allowed MIME types and file signatures (magic numbers), not just extensions.
• Use file locks for concurrent access when needed to avoid race conditions (advisory/mandatory locks depending on platform).
• Limit file sizes and resource usage to prevent denial-of-service via huge uploads.
• Store secrets (API keys, DB passwords) securely — not in plain text files committed into source control. Use environment variables, secret managers or encrypted files.
• Set correct OS file permissions: owner-only for sensitive files; use chown/chmod/chattr as appropriate.
• Encrypt sensitive files at rest (AES-256 or similar) and use secure channels (SFTP/HTTPS/TLS) for transfer.
• Use checksums/hashes (e.g., SHA-256) to verify integrity after transfer or backup.
• Rotate logs and use log retention policies; archive older files and delete securely if required.
• Use secure temporary file APIs (e.g., mkstemp) instead of predictable names to avoid symlink attacks.
• Keep software and libraries up-to-date to fix filesystem-related vulnerabilities.

Security-specific measures

• Access control: combine user/group permissions with ACLs for fine-grained control.
• Authentication & authorization: ensure only authenticated users can access protected files and that their actions are authorized.
• Encryption: encrypt files at rest and use TLS for network transfer. Protect encryption keys using key management systems.
• Integrity checks: maintain and verify hashes; maintain audit logs of file changes.
• Secure deletion: overwrite or use secure-delete utilities if sensitive data must be unrecoverable (note: some filesystems and SSDs complicate secure erase).

Example secure workflow for saving user-uploaded files

1. Authenticate user and check authorization.
2. Validate filename (remove path separators) and generate a safe internal name or UUID.
3. Check file size and MIME type and verify magic number.
4. Write to a secure temporary file using a safe API (mkstemp), fsync to disk, then atomically rename to destination.
5. Set strict file permissions and, if required, encrypt the file.
6. Log the upload event (who, when, size, hash) for audit.

Common pitfalls to avoid

• Leaving files open or not handling exceptions — leads to resource leaks.
• Relying on file extensions to determine type — attackers can rename malicious files.
• Using predictable temp filenames — opens symlink or race attacks.
• Storing secrets in repository or world-readable files.
• Not rotating or backing up data — risk of permanent loss.

Summary
Combine secure coding (input validation, atomic operations, exception handling) with system controls (permissions, encryption, logging, backups) to create robust and secure file handling systems.