Chapter 16 — Probability

Random experiment: A random (or stochastic) experiment is a process or action which can be repeated under the same conditions and whose exact result cannot be predicted with certainty in advance. Examples: tossing a coin, rolling a die, measuring the lifetime of a bulb.

Outcome (Elementary outcome): An outcome is a single possible result of a random experiment. Each repetition produces exactly one outcome. Notation: use a lowercase letter (e.g., ω) or list form (e.g., 3 when a die shows 3).

Sample space (S): The sample space of an experiment is the set of all possible outcomes. It is usually denoted by S or Ω. Example notations: S = {H, T} for a coin toss; S = {1,2,3,4,5,6} for a fair die; S = [0, 60] seconds for measuring waiting time up to one minute.

Types of sample spaces

• Finite: S has a finite number of outcomes (e.g., die has 6 outcomes).
• Countably infinite (discrete): S has infinitely many countable outcomes (e.g., number of tosses until first head).
• Uncountable (continuous): S is an interval of real numbers (e.g., all real values between 0 and 1).

Events: An event is any subset of the sample space (one or more outcomes). An elementary event contains exactly one outcome. The impossible event is ∅ and the sure event is S.

Set notation and basic relationships: If E and F are events (subsets of S), then E ∪ F (union) and E ∩ F (intersection) are also events. The complement of E is E^c = S \ E.

Understanding sample spaces and outcomes is the first step to assigning probabilities and solving probability problems: list S clearly, identify the favourable outcomes for the event of interest, then compute probabilities (using appropriate rules) depending on whether outcomes are equally likely or not.

Definition: In probability, an event is any collection (subset) of possible outcomes of a random experiment. The set of all possible outcomes is the sample space S. An event E is therefore E \u2286 S.

Types of events:

• Simple (Elementary) event: Contains exactly one outcome (e.g., getting '4' on a die).
• Compound event: Contains two or more outcomes (e.g., getting an even number on a die).
• Sure (Certain) event: The event S that always occurs (P(S)=1).
• Impossible event: The empty set Ø that never occurs (P(Ø)=0).
• Complementary event: For event A, A' (or A^c) is the set of outcomes in S not in A; P(A')=1−P(A).
• Mutually exclusive (disjoint) events: A and B are mutually exclusive if A∩B=Ø (they cannot both occur).
• Exhaustive events: A collection of events whose union is the whole sample space S.
• Independent events: Two events A and B are independent if the occurrence of one does not affect the probability of the other: P(B|A)=P(B).

Set notation and relations: Events are handled with set operations: union (A \u222a B), intersection (A \u2229 B), complement (A'), difference (A \u2216 B). Venn diagrams are commonly used to visualize these relations.

Probability of an event (equally likely outcomes): If outcomes are equally likely and S is finite, P(E) = n(E)/n(S), where n(E) is the number of favourable outcomes and n(S) the total outcomes.

Logic and laws: De Morgan's laws for events: (A \u2229 B)' = A' \u222a B' and (A \u222a B)' = A' \u2229 B'. These help simplify complements of unions/intersections.

Connection to conditional probability: For general events, intersection and conditional probability relate by P(A \u2229 B) = P(A)P(B|A). For independent events this reduces to P(A \u2229 B) = P(A)P(B).

Understanding events and their relations is the foundation for computing probabilities of complex situations by breaking them into unions, intersections, complements and sequences.

Definition. If a random experiment has a finite sample space S with n(S) equally likely outcomes and E is an event with n(E) favourable outcomes, then the probability of E is

P(E) = n(E) / n(S)

Here "equally likely" means every elementary outcome has the same chance of occurring. The classical definition reduces probability to counting favourable outcomes divided by total outcomes.

How to apply (step-by-step):

• Identify the sample space S and verify outcomes are equally likely.
• Count total outcomes n(S).
• Describe event E and count favourable outcomes n(E) (use combinations/permutations if needed).
• Compute P(E) = n(E)/n(S) and simplify.

Important properties (consequences):

• 0 ≤ P(E) ≤ 1 for any event E.
• P(S) = 1 and P(Ø) = 0.
• If A and B are mutually exclusive (disjoint), P(A ∪ B) = P(A) + P(B).
• For any event A, P(A^c) = 1 − P(A), where A^c is the complement.
• General addition rule: P(A ∪ B) = P(A) + P(B) − P(A ∩ B).

Counting tools. For many problems n(E) and n(S) are found using permutations (P) and combinations (C): e.g. if choosing k objects from n without order, use C(n, k) = n!/(k!(n−k)!).

Limitations. The classical definition applies only when (a) the sample space is finite and (b) elementary outcomes are equally likely. It does not apply directly when outcomes have different probabilities or when the sample space is infinite.

Definition: Experimental (or empirical) probability of an event E is the ratio of the number of times the event occurs to the total number of trials performed. It is also called the relative frequency of the event. If an experiment is repeated n times and the event E occurs f(E) times, then the experimental probability of E is P(E) ≈ f(E)/n.

Procedure:

• Specify the experiment and the event E.
• Perform the experiment n times under the same conditions.
• Count f(E), the number of trials in which E occurs.
• Compute the relative frequency P(E) = f(E)/n. This gives an empirical estimate of the probability of E.

Key idea — Law of Large Numbers: As the number of trials n increases, the experimental probability f(E)/n tends to approach the theoretical probability P(E) (if the theoretical probability exists and the trials are independent and identically conducted). In practice, larger n usually gives a more reliable estimate.

Comparison with theoretical probability: Theoretical probability is computed from known equally likely outcomes (for example, 1/6 for a fair die face) without performing experiments. Experimental probability estimates the probability by actual repeated trials; it may differ from the theoretical value for small n but should get closer as n grows.

Sources of error / considerations: Small sample size, biased or non-random trials, dependence between trials (e.g., drawing without replacement) and measurement errors can make the experimental probability deviate from the theoretical value.

Uses: Estimating probabilities when theoretical models are hard or impossible (e.g., reliability testing, medicine, weather, quality control), validating theoretical models, and teaching probability concepts.

Definition (for equally likely outcomes): If a random experiment has a finite sample space S with equally likely outcomes and an event A has n(A) favourable outcomes, then the probability of A is P(A) = n(A)/n(S).

Fundamental properties:

• Non-negativity: For any event A, P(A) ≥ 0. Probabilities cannot be negative.
• Normalization (total probability): P(S) = 1, where S is the sample space (something in S must happen).
• Probability of impossible event: P(∅) = 0, where ∅ is the null (impossible) event.
• Complement rule: For an event A, its complement A' (event 'A does not occur') satisfies P(A') = 1 − P(A).
• Additivity for disjoint events: If A and B are mutually exclusive (A ∩ B = ∅), then P(A ∪ B) = P(A) + P(B).
• General addition rule: For any two events A and B, P(A ∪ B) = P(A) + P(B) − P(A ∩ B). This removes double counting of outcomes in A ∩ B.
• Monotonicity (inclusion): If A ⊆ B then P(A) ≤ P(B).
• Bounds: From the above, 0 ≤ P(A) ≤ 1 for every event A.

Short derivations / remarks:

• From P(S)=1 and A ∪ A' = S with A and A' disjoint, P(A) + P(A') = 1 ⇒ P(A') = 1 − P(A).
• For general A,B, write A ∪ B = A + (B \ A) where (B \ A) is disjoint from A; hence P(A ∪ B) = P(A) + P(B \ A) = P(A) + P(B) − P(A ∩ B).
• Inequality P(A ∪ B) ≤ P(A) + P(B) follows because P(A ∩ B) ≥ 0.

Why these matter: These properties form the algebra of probability: they let you combine events, compute complements, and ensure consistency (probabilities stay between 0 and 1 and total 1 over the sample space).

What is the addition rule?
The addition rule gives the probability that at least one of two (or more) events occurs — in other words, the probability of the union of events.

Two events (general case):
For any two events A and B in the same sample space,

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

Why we subtract P(A ∩ B): When we add P(A) and P(B) we count outcomes in A ∩ B twice, so we subtract P(A ∩ B) once to correct the double counting.

Special case — mutually exclusive events:
If A and B are mutually exclusive (disjoint), then A ∩ B = ∅ and P(A ∩ B) = 0, so

P(A ∪ B) = P(A) + P(B) (if A and B are mutually exclusive)

Complement form:
It is often useful to write the union probability using complements:

P(A ∪ B) = 1 − P(A' ∩ B')
(the probability that not both A and B fail)

Extension to three events (inclusion–exclusion):
For events A, B, C,

P(A ∪ B ∪ C) = P(A)+P(B)+P(C) − P(A∩B) − P(A∩C) − P(B∩C) + P(A∩B∩C)

This pattern generalises to more events (the inclusion–exclusion principle).

Short proof idea for two events:
Write B as (B \ A) ∪ (A ∩ B). Then A ∪ B = A ∪ (B \ A), where A and (B \ A) are disjoint. So P(A ∪ B) = P(A) + P(B \ A) = P(A) + [P(B) − P(A ∩ B)].

Use in practice:
To find P(A ∪ B) you need P(A), P(B) and the overlap probability P(A ∩ B). If the overlap is unknown, find it from data or by counting the outcomes in the intersection.

What it is: In probability, counting techniques are used to find the number of possible outcomes in the sample space and the number of favourable outcomes for an event. When all outcomes are equally likely, probability = (number of favourable outcomes) / (number of total outcomes). Counting techniques help compute these numbers efficiently.

Key ideas and how to choose a technique:

• Fundamental principle of counting (multiplication rule): If a task can be done in a sequence of steps with m choices for step 1 and n choices for step 2, total choices = m × n (extendable to more steps).
• Order matters → Permutations: Use permutations when different orders are distinct. nPr = n! / (n − r)! gives the number of ordered arrangements of r objects chosen from n.
• Order doesn’t matter → Combinations: Use combinations when only the selection matters (not order). nCr = n! / (r!(n − r)!) gives the number of ways to choose r objects from n.
• Repetition allowed: If repetition is allowed and order matters, use n^r. If repetition allowed but order doesn’t matter, use combinations with repetition: C(n + r − 1, r).
• Identical objects: For permutations of n objects with groups of identical items, total = n!/(n1! n2! …).
• Complementary counting: Sometimes it is easier to count the complement and subtract from the total.

Applying to probability: 1) Identify the sample space and whether outcomes are equally likely. 2) Use the appropriate counting method to find |S| and |E|. 3) Compute P(E) = |E| / |S|. If outcomes are not equally likely, count or compute probabilities for each outcome (often using tree diagrams or binomial formula).

Common pitfalls: confusing when order matters, forgetting to account for replacement (with/without), and not checking whether outcomes are equally likely.

This topic summarizes frequently used probability models (both discrete and continuous), when to use each model, their main properties and how they relate. Each model is presented with its defining formula (PMF or PDF), expectation and variance, conditions/assumptions, and a short intuitive description.

Discrete models

• Discrete uniform: all outcomes in a finite set of size N are equally likely. Good for fair dice, shuffled cards, lottery numbers.
• Bernoulli: a single trial with two outcomes: success (probability p) or failure (probability 1-p). Use for a single yes/no experiment (coin toss, pass/fail).
• Binomial: number of successes in n independent Bernoulli trials with same p. Use when you have fixed number of repeated independent trials (e.g., 10 coin tosses, number of defective items in a sample of n).
• Geometric: number of trials until the first success (support 1,2,...). Use when you wait for the first success and trials are independent with same p.
• Hypergeometric: number of successes in n draws without replacement from a finite population containing K successes and N−K failures. Use when sampling without replacement (e.g., drawing cards without replacement).
• Poisson: models count of rare events in a fixed interval if events occur independently and average rate λ is constant. Use for arrivals, rare defects, phone calls per hour when events are rare and many potential opportunities.

Continuous models

• Continuous uniform: every point in interval [a,b] is equally likely (density 1/(b−a)). Use for a random point on a length, random time in an interval when all instants are equally likely.
• Exponential: models waiting time between independent Poisson events with rate λ (memoryless property). Use for lifetimes without aging (constant hazard), e.g., time between phone calls.
• Normal (Gaussian): bell-shaped symmetric distribution determined by mean μ and variance σ2. Many sums of small independent effects are approx. normal (central limit theorem). Use for measurement errors, heights, scores (when approximately symmetric and unimodal).

Key relations and approximations:

• Binomial(n,p) ≈ Poisson(λ = np) when n is large and p is small (np moderate).
• Binomial(n,p) ≈ Normal(mean=np, variance=np(1−p)) when n is large and np and n(1−p) are both ≥ about 5 (use continuity correction for discrete-to-continuous approximation).
• Exponential is the continuous analogue of the geometric distribution (both are memoryless).

Understanding which model fits a situation comes from checking assumptions: discrete vs continuous, independence, identical trials, replacement vs no replacement, constant rate, rarity of events, etc.

Overview

Problem-solving in probability combines logical reasoning, counting methods and choice of the right model. A reliable approach follows Polya's four steps: (1) understand the problem, (2) devise a plan (choose counting/conditioning/diagrams), (3) carry out the plan (compute probabilities), and (4) review and interpret the result.

Key strategies

• Define the sample space clearly (equally likely outcomes when appropriate). Write S and count n(S).
• Use counting techniques: permutations and combinations for arrangements and selections; multiplication principle for sequential choices.
• Use complements: when "at least one" or "none" type events appear, compute P(A)=1−P(A complement) if easier.
• Condition and partition: split complicated events into simpler conditional cases (use law of total probability).
• Apply Bayes' theorem for reversed conditional questions (find cause given effect).
• Test independence: if P(A∩B)=P(A)P(B), events are independent; otherwise use conditional probabilities.
• Use symmetry to reduce counting (identical objects, identical roles).
• Diagrams and representations: tree diagrams for sequential/conditional problems, Venn diagrams for unions/intersections, sample-space grids for dice/cards.
• Check edge cases and sanity: probabilities must lie in [0,1]; compare with simple bounds (e.g., complement or symmetry).

Applications

Probability techniques apply to games of chance (dice, cards), quality control (defective rates), medical testing (sensitivity/specificity and Bayesian updating), reliability of systems, risk assessment, and simple genetics (Punnett-square probabilities). In modelling, translate the real situation into an appropriate experiment, identify events of interest, and use counting/conditioning to compute probabilities.

How to choose a method

1. If outcomes are equally likely and finite, counting (n(A)/n(S)) is often simplest.
2. If outcomes occur in sequence or depend on previous draws, consider conditional probability and trees.
3. For "at least one" problems use complements.
4. For questions asking "given that B occurred, what's the chance of A?" use P(A|B) and Bayes if reversing conditions.

Following these structured steps reduces errors and makes reasoning clear and checkable.