Can ten people tell you what millions think?
A diet study with twelve participants; a survey of five friends who all agree. Studying a small group to learn about a large one is normal and necessary — but only if the group is big enough, and only if it actually resembles the population it is standing in for.
"Study of 12 people proves new diet works!" "I surveyed my 5 friends - they all agree!" The problem? SAMPLE SIZE (too few) and REPRESENTATIVENESS (not diverse enough). Let's learn to spot weak samples!
SAMPLE = small group studied to learn about a LARGER population. Can't ask all Americans their opinion? Ask a sample! But for reliable results: (1) Sample must be LARGE enough, (2) Sample must be REPRESENTATIVE (match the larger group's diversity)!
TOO SMALL = unreliable! Flip a coin 3 times: might get 3 heads (100%!). Flip 1,000 times: approaches 50/50. LARGER samples reduce the impact of random chance and outliers. General rule: Hundreds minimum for population studies, thousands better!
REPRESENTATIVE SAMPLE = reflects the diversity of full population! Surveying 1,000 college students about ALL Americans? Not representative (too young, educated, etc). Need mix of: ages, locations, backgrounds, income levels. SIZE ≠ quality if sample is biased!
Be skeptical when: • Sample size under 100 (for population claims), • Only from one location/group, • Self-selected (online polls - only motivated people respond!), • No info about HOW sample was chosen. Always ask: "How many? Who exactly? How were they selected?"
Good samples must be both LARGE ENOUGH and REPRESENTATIVE of the population!
Sample size:
• Too small: Random chance dominates (unreliable)
• Large enough: Patterns emerge, chance evens out
• Rule of thumb: 100s minimum, 1000s+ ideal for population studies
Representativeness:
Sample should MIRROR the population in key characteristics:
• Age distribution
• Geographic spread
• Income levels
• Education levels
• Gender balance
• Ethnic diversity
Bad sampling examples:
• "I asked 10 people at the mall" (tiny + biased location)
• "1,000 Twitter users said..." (self-selected, not representative)
• "Study of Harvard students shows..." (not representative of all students)
Good sampling:
• Random selection from full population
• Stratified (ensuring diverse representation)
• Large enough for statistical reliability
• Transparent about methodology
Critical questions:
1. How many people?
2. Who were they exactly?
3. How were they chosen?
4. Do they represent the full population?
Remember: 1,000 biased people < 100 well-chosen people!
🌱 Story Seed
"This product has 100% 5-star reviews!" "How many reviews?" "Three." "Ah - three people loved it. That doesn't mean YOU will. Let's find one with hundreds of reviews to see the real pattern."