Chapter 13 — Probability

1. Random experiment, outcome, sample space and event
A random experiment is an action or process with well-defined possible outcomes but uncertain result (e.g., tossing a coin). Each possible result is an outcome. The set of all possible outcomes is the sample space S. An event is any subset of S (one or more outcomes). Examples of events: A = “get an even number” when rolling a die.

2. Types of events

• Elementary event: a single outcome (e.g., {2} when rolling a die).
• Impossible event: the empty set Ø.
• Sure (certain) event: the sample space S.
• Complementary event: A' (outcomes in S not in A).
• Mutually exclusive (disjoint) events: A and B are disjoint if A ∩ B = Ø.
• Exhaustive events: events whose union is S.

3. Classical (equally likely) probability
If an experiment has a finite number of equally likely outcomes, probability of an event A is

P(A) = (number of favourable outcomes for A) / (total number of outcomes in S).

4. Axiomatic definition (Kolmogorov)
A function P defined on events of S is a probability measure if it satisfies:

• Non-negativity: P(A) ≥ 0 for every event A.
• Normalization: P(S) = 1.
• Countable additivity: If A1, A2, ... are mutually exclusive, then P(union Ai) = sum P(Ai).

5. Basic consequences / rules (derived)

• P(Ø) = 0 and 0 ≤ P(A) ≤ 1 for every event A.
• Complement rule: P(A') = 1 − P(A).
• Monotonicity: If A ⊆ B then P(A) ≤ P(B).
• Addition rule (two events): P(A ∪ B) = P(A) + P(B) − P(A ∩ B). If A and B are disjoint, P(A ∪ B) = P(A) + P(B).
• Inclusion–exclusion (three events): P(A ∪ B ∪ C) = P(A)+P(B)+P(C) − P(A∩B) − P(B∩C) − P(C∩A) + P(A∩B∩C).
• Union bound (Boole’s inequality): P(∪_{i=1}^n A_i) ≤ Σ_{i=1}^n P(A_i).

6. Multiplication / conditional rule (basic)
Conditional probability of A given B (if P(B) > 0) is defined as P(A|B) = P(A ∩ B) / P(B). Rearranging gives the multiplication rule:

P(A ∩ B) = P(B) × P(A|B) = P(A) × P(B|A).

7. General multiplication for many events
For events A1, A2, ..., An,

P(A1 ∩ A2 ∩ ... ∩ An) = P(A1) × P(A2|A1) × P(A3|A1 ∩ A2) × ... × P(An|A1 ∩ ... ∩ A_{n−1}).

Summary
These preliminaries set the language and the basic algebra of probability: describe outcomes (S and events), compute probabilities under equally likely assumption, and use axioms to derive complement, addition, multiplication and conditional rules that are applied to real problems.

Definition: The conditional probability of event A given event B (written P(A|B)) is the probability that A occurs when we know that B has occurred. Formally, for P(B) > 0,

P(A|B) = P(A ∩ B) / P(B).

Interpretation: Knowing B restricts the sample space to B. The conditional probability is the proportion of outcomes in B that also lie in A.

Key related results:

• Multiplication rule: P(A ∩ B) = P(B) P(A|B) = P(A) P(B|A).
• General multiplication for n events: P(A1 ∩ A2 ∩ ... ∩ An) = P(A1) P(A2|A1) P(A3|A1 ∩ A2) ... P(An|A1 ∩ ... ∩ A{n-1}).
• Law of total probability: If {B1, B2, ..., Bn} is a partition of the sample space with P(Bi) > 0, then P(A) = sum_{i=1..n} P(A|Bi) P(Bi).
• Bayes' theorem: For a partition {B1,...,Bn} and any j, P(Bj|A) = P(A|Bj) P(Bj) / sum_{i=1..n} P(A|Bi) P(Bi).
• Independence: A and B are independent iff P(A|B) = P(A) (equivalently P(A ∩ B) = P(A) P(B)).

How to solve problems (Class 12 tips):

1. Identify events A and B and what is given (P(B) or conditional probabilities).
2. Decide whether to use direct definition, multiplication rule, tree diagram, or Bayes' theorem.
3. If there is sequential action (draws, tests), draw a tree showing conditional branches and attach probabilities to branches.
4. If data are in a table, compute conditional probabilities by dividing the appropriate cell by the corresponding row or column total.

Usage: Conditional probability models updating of information (e.g., test result alters belief about disease), dependence between sequential draws (with or without replacement), and problems with partitions of the sample space.

Definition and idea: The multiplication theorem relates the probability of the intersection of events to conditional probabilities. If A and B are two events with P(A) > 0, the conditional probability of B given A is

P(B|A) = P(A ∩ B) / P(A).

Rearranging gives the basic multiplication theorem for two events:

P(A ∩ B) = P(A) · P(B | A)

This means the probability that both A and B occur equals the probability that A occurs times the probability that B occurs given that A has already occurred.

Extension to three events: For events A, B and C (with P(A) > 0 and P(A ∩ B) > 0),

P(A ∩ B ∩ C) = P(A) · P(B | A) · P(C | A ∩ B)

In general, for n events A1, A2, ..., An,

P(A1 ∩ A2 ∩ ... ∩ An) = P(A1) · P(A2 | A1) · P(A3 | A1 ∩ A2) · ... · P(An | A1 ∩ ... ∩ A(n-1))

Special case — Independence: If A and B are independent, P(B | A) = P(B), so the multiplication formula becomes

P(A ∩ B) = P(A) · P(B)

(For mutually independent events this extends: P(A1 ∩ ... ∩ An) = Π P(Ai).)

Important notes: (1) Conditional probabilities require the conditioning event to have positive probability. (2) Independence is a stronger condition than P(A ∩ B) = P(A)P(B); for more than two events pairwise independence does not imply mutual independence.

Definition: Two events A and B in a probability space are said to be independent if the occurrence of one does not change the probability of the other. Formally, A and B are independent iff

P(A ∩ B) = P(A) · P(B)

Equivalently, if P(A) > 0, independence can be stated as P(B | A) = P(B), i.e. the conditional probability of B given A equals the (unconditional) probability of B.

Explanation and intuition: If knowing that A occurred gives no information about whether B occurs, the two events are independent. For independent events the joint probability factorizes into the product of individual probabilities. Independence is a statement about how probabilities relate, not about causal relationships.

Pairwise vs Mutual (joint) independence: For three events A, B, C:

• Pairwise independence: each pair is independent: P(A∩B)=P(A)P(B), P(A∩C)=P(A)P(C), P(B∩C)=P(B)P(C).
• Mutual (or collective) independence: every subcollection (including the triple) satisfies the product rule. In particular, besides pairwise equalities, we must have P(A∩B∩C)=P(A)P(B)P(C).

Pairwise independence does not imply mutual independence in general. A standard counterexample is given by three events that are pairwise independent but whose triple intersection does not equal the product of individual probabilities.

Properties:

• If A and B are independent then A and B complement (B^c) are independent, and A^c is independent of B and of B^c.
• If P(A)=0 or P(B)=0 then P(A∩B)=0, so independence holds trivially (degenerate case).
• If P(B)=1 then B is independent of every event (also degenerate).
• For independent events the multiplication rule extends: if A1, A2, ..., An are mutually independent, then P(A1 ∩ A2 ∩ ... ∩ An) = Π P(Ai).

How to test independence: Compute P(A), P(B) and P(A∩B). If P(A∩B) equals P(A)P(B) (within rounding), A and B are independent. Or check if P(B|A)=P(B) (provided P(A)>0).

Meaning: The rule of total probability (partition method) gives the probability of an event A by splitting the sample space into a finite or countable collection of mutually exclusive and exhaustive events B1, B2, …, Bn (a partition). If we know the probabilities of each Bi and the conditional probabilities P(A|Bi), the total probability of A is the weighted sum of these conditional probabilities.

Conditions:

• Events B1, B2, …, Bn form a partition: they are pairwise disjoint (Bi ∩ Bj = ∅ for i ≠ j) and their union is the whole sample space (⋃i Bi = S).
• For each i, P(Bi) > 0 so conditional probability P(A|Bi) is defined.

Statement / Formula (one-line):

P(A) = Σ_{i=1}^n P(A | B_i)·P(B_i)

Derivation (idea): Because the Bi partition S, A = A∩S = A∩(⋃i Bi) = ⋃i (A∩Bi). These events A∩Bi are disjoint, so P(A) = Σi P(A∩Bi). But P(A∩Bi) = P(A|Bi)·P(Bi). Summing gives the total probability formula.

When to use: Use when A is easier to analyze by conditioning on which part (Bi) of the partition occurred — typical when there are different sources, boxes, machines, population groups, etc.

Connection to Bayes' theorem: Once you have P(A) from the total probability rule, you can compute posterior probabilities P(Bj|A) using Bayes' formula:

P(B_j | A) = [P(A | B_j)·P(B_j)] / Σ_{i=1}^n P(A | B_i)·P(B_i)

Example structure and steps:

1. Identify a natural partition {B1, …, Bn} of the sample space.
2. Find P(Bi) for each i (prior probabilities).
3. Find P(A|Bi) for each i (conditional probabilities).
4. Compute P(A) = Σ P(A|Bi)P(Bi).

What is Bayes' theorem?
Bayes' theorem gives a way to update the probability of a hypothesis (an event A_i) after observing new evidence B. It computes the posterior probability P(A_i|B) from prior probabilities P(A_i) and likelihoods P(B|A_i).

Setup and statement.
Let {A1, A2, ..., An} be a partition of the sample space S (they are mutually exclusive and exhaustive) and let B be an event with P(B) > 0. Then for any i = 1, 2, ..., n:

P(A_i | B) = \dfrac{P(A_i)\,P(B | A_i)}{\sum_{k=1}^n P(A_k)\,P(B | A_k)}

Derivation (brief).
By definition P(A_i | B) = P(A_i ∩ B)/P(B). But P(A_i ∩ B) = P(A_i)P(B | A_i). Also P(B) = Σ_k P(A_k ∩ B) = Σ_k P(A_k)P(B | A_k). Substitute to get the formula above.

Interpretation.
P(A_i) is the prior probability of hypothesis A_i before seeing B. P(B | A_i) is the likelihood of observing B when A_i is true. The denominator normalizes over all hypotheses so the posteriors sum to 1. Bayes' theorem is used to update beliefs when new evidence arrives.

When to use.
Use Bayes' theorem whenever you have several mutually exclusive possible causes (A_i) and you observe an effect B — you want the probability of each cause given the effect.

Solution steps for problems.

• Identify the partition events A1,...,An and the observed event B.
• Write prior probabilities P(A_i) and likelihoods P(B|A_i).
• Compute P(B)=Σ_k P(A_k)P(B|A_k).
• Compute P(A_i|B)=P(A_i)P(B|A_i)/P(B).

Overview: In probability problems you repeatedly use a small set of techniques: counting (classical approach), complement rule, conditioning (including total probability and Bayes), independence, symmetry, and common distribution formulas (binomial, hypergeometric). The usual workflow is: (1) model the experiment and sample space, (2) choose an appropriate counting/probability model (equally likely outcomes, conditional model, or distribution), (3) simplify using complements or symmetry where possible, and (4) compute using formulas or tree/Venn diagrams and check consistency (probabilities between 0 and 1, total = 1 when needed).

Key techniques:

• Counting / Classical method: If outcomes are equally likely, P(E)=#(favourable)/#(total). Use permutations and combinations to count outcomes.
• Complement rule: Often easier to compute P(E') and use P(E)=1−P(E'). This is useful for "at least one" problems.
• Conditioning and tree diagrams: For sequential experiments (with/without replacement) or staged processes, build a tree and use conditional probabilities: P(A∩B)=P(A)P(B|A).
• Total probability: When the sample space is partitioned into B1,B2,..., use P(A)=sum_i P(Bi)P(A|Bi).
• Bayes' theorem: To reverse conditioning: P(Bj|A)=P(Bj)P(A|Bj) / sum_i P(Bi)P(A|Bi). Common for diagnosis/testing problems.
• Independence: If events A and B are independent, P(A∩B)=P(A)P(B) and P(A|B)=P(A). Always check independence, don’t assume it.
• Symmetry: When all outcomes or players are symmetric, you can assign equal probabilities or use symmetry to reduce counting.

Recurring problem types & strategies:

• Urn problems (with/without replacement): Model with combinations (without replacement -> hypergeometric), or with Binomial/independent trials (with replacement).
• At least one successes: Use complement: P(at least one) = 1 − P(none).
• Sequential draws: Use tree diagrams and conditional probabilities; multiply along branches to get joint probabilities and sum branches for a final event.
• Rare-event or diagnostic problems: Use total probability to find probability of a test result, then Bayes to find the probability a subject actually has the condition given the test result.
• Card / dice problems: Use equally likely assumptions, count favourable outcomes by combinatorics, exploit symmetry to simplify.
• Independence checks: Verify P(A∩B) = P(A)P(B) before using independence-based shortcuts.

Worked strategy example (short): "Probability of at least one six in 4 fair dice" — Instead of counting 1,2,... successes, use complement: P(no six in one die)=5/6 so P(no six in 4)= (5/6)^4, hence P(at least one six)=1−(5/6)^4.

Practical tips:

• Draw the sample space diagram or Venn diagram for events; for multi-stage problems draw a tree and label branch probabilities.
• Use complements when direct counting is hard ("at least one", "none").
• When replacement is present, trials are independent; when without replacement, use hypergeometric counting.
• For conditional problems, clearly state the conditioning event and restrict the sample space accordingly.

This topic collects the most important distinctions and common mistakes students make in probability. Each point below gives what to check, why it matters, and a short rule to remember.

1. Mutually exclusive vs independent: Mutually exclusive (disjoint) events cannot occur together: P(A&B)=0. Independent events have P(A&B)=P(A)P(B). If P(A)>0 and P(B)>0, mutual exclusivity implies dependence (they cannot be independent unless one has zero probability). Check by computing P(A&B) and P(A)P(B).
2. Addition rule and overlap: For any A,B: P(A∪B)=P(A)+P(B)−P(A&B). Using P(A)+P(B) without subtracting the intersection is a common error unless events are mutually exclusive.
3. Intersection vs conditional vs multiplication rule: P(A&B)=P(A)P(B|A)=P(B)P(A|B). Do not confuse P(A|B) with P(B|A). Bayes' theorem reverses conditioning but requires correct priors.
4. Zero-probability conditioning: P(A|B) is undefined if P(B)=0. Never condition on an impossible event.
5. Complement (at least one) trick: Events like "at least one success" are often easier computed via complement: P(at least one)=1−P(none). This avoids messy inclusion–exclusion.
6. With vs without replacement (sampling): With replacement often yields independent trials (use binomial). Without replacement changes probabilities each draw (use hypergeometric or conditional counting). Mixing these up leads to wrong answers.
7. Classical definition requires equally likely outcomes: P(E)=#favourable/#total only when all sample points are equally likely. For biased processes, use conditional or distribution-based formulas instead.
8. Discrete vs continuous: For continuous variables P(X=x)=0. Probabilities come from integrating the pdf over an interval; do not treat pdf values as probabilities.
9. Pairwise vs mutual independence: Events can be pairwise independent (every pair independent) but not mutually independent (joint independence fails). Always check the full joint condition for more than two events.
10. Expectation and variance pitfalls: E[g(X)] generally ≠ g(E[X]) (nonlinear transformations mistaken). Var(X+Y)=Var(X)+Var(Y)+2Cov(X,Y); you can omit the covariance term only when X and Y are independent (Cov=0).
11. Counting: permutations vs combinations: Check whether order matters. Use nPr = n!/(n−r)! when order matters, and nCr = n!/(r!(n−r)!) when it does not. Many probability errors come from wrong counting.

Quick checks before answering: (1) Are outcomes equally likely? (2) Are trials independent or not? (3) Is conditioning on an event with nonzero probability? (4) Are you asked for intersection, union, or conditional probability?