Chapter 4.3: Proportion Tests
Chapter 4.3: Proportion Tests
Overview
Proportion tests are used when your data consists of categories or yes/no outcomes, and you want to test claims about percentages or success rates. Unlike t-tests which compare means, proportion tests compare percentages.
The Core Question:
“Is this percentage/success rate significantly different from what we expected or from another group?”
Real-World Applications:
- Marketing & A/B Testing: Did the new ad campaign increase conversion rates?
- Quality Control: Is the defect rate below 5%?
- Medical Research: Is the cure rate for Treatment A higher than Treatment B?
- Politics: Do different demographics support a candidate at different rates?
- Product Testing: Do users prefer Feature A over Feature B?
When to Use Proportion Tests:
Use proportion tests when your data is:
- Binary/Categorical: Yes/No, Success/Failure, Click/No Click, Pass/Fail
- Count-based: Number of successes out of total trials
- Percentage-based: Conversion rates, success rates, approval ratings
Examples:
- ✅ “60 out of 100 customers were satisfied” → Proportion = 0.60
- ✅ “15% of products were defective” → Proportion = 0.15
- ✅ “450 out of 1000 clicked the ad” → Proportion = 0.45
NOT for proportion tests:
- ❌ Average customer spending ($45.50) → Use t-test
- ❌ Mean test score (78.5) → Use t-test
- ❌ Average delivery time (23.4 minutes) → Use t-test
Two Types of Proportion Tests:
1. One-Sample Proportion Test
Compare your sample proportion to a claimed or expected value.
Example: “A company claims 90% customer satisfaction. You survey 200 customers and find 85% are satisfied. Is the company’s claim accurate?”
2. Two-Sample Proportion Test
Compare proportions between two independent groups.
Example: “Do men (20% click rate) and women (30% click rate) click on ads at different rates?”
Key Differences from T-Tests:
| Aspect | T-Test | Proportion Test |
|---|---|---|
| Data Type | Continuous (means) | Categorical (percentages) |
| Example | Average height: 170 cm | 60% are tall |
| Test Statistic | t-statistic | z-statistic |
| Distribution | t-distribution | Normal distribution |
| Sample Size | Works with small samples | Needs larger samples (np ≥ 10) |
What You’ll Learn:
- One-sample proportion test: Test if a percentage matches a claimed value
- Two-sample proportion test: Compare percentages between two groups
- Sample size requirements: When proportion tests are valid
- Practical interpretation: What the results mean for decision-making
Table of Contents
- Introduction - Testing Percentages
- One-Sample Proportion Test
- Two-Sample Proportion Test
- Python Implementation - Let’s Code It!
- Practice Exercises - Your Turn!
1. Introduction - Testing Percentages
The Big Picture (Explained Simply)
So far, we’ve been testing averages (means):
- Chapter 4.1: Is the average chocolate bar weight 100g?
- Chapter 4.2: Do men and women have different average salaries?
Now we’re going to test percentages (proportions):
- Is the success rate really 50%?
- Do more women than men prefer product A?
- Has customer satisfaction improved from 60% to 75%?
What’s a Proportion?
A proportion is just a fancy word for a percentage expressed as a decimal:
- 50% = 0.50
- 75% = 0.75
- 23% = 0.23
Examples of proportions:
- 🎯 60% of customers are satisfied (proportion = 0.60)
- 🗳️ 45% of voters support Candidate A (proportion = 0.45)
- 🎓 80% of students passed the exam (proportion = 0.80)
- 🏥 15% of patients experienced side effects (proportion = 0.15)
Two Types of Proportion Tests
Type 1: One-Sample Proportion Test
Compare ONE sample proportion to a claimed value.
Example: A coin should land on heads 50% of the time. You flip it 100 times and get 60 heads. Is the coin fair?
- Claimed proportion: 50% (0.50)
- Your sample: 60% (0.60)
- Question: Is this difference just random luck, or is the coin biased?
Type 2: Two-Sample Proportion Test
Compare TWO sample proportions to each other.
Example: Do men and women click on ads at different rates?
- Men: 100 out of 500 clicked (20%)
- Women: 150 out of 500 clicked (30%)
- Question: Is this difference real, or just random variation?
Real-Life Examples
One-Sample:
- 🎲 Is this die fair? (Should be 1/6 = 16.67% for each number)
- 📊 Is our conversion rate really 10%?
- 🏥 Is the cure rate at least 80%?
Two-Sample:
- 👨👩 Do men and women have different click-through rates?
- 💊 Is Drug A more effective than Drug B?
- 🎯 Did our marketing campaign improve conversion rates?
2. One-Sample Proportion Test
What is a One-Sample Proportion Test? (Simple Explanation)
Imagine you’re a quality control manager at a factory. The company claims that only 5% of products are defective. You inspect 200 random products and find 15 defective ones (7.5%).
The Question: Is the defect rate really 5%, or is it higher?
This is a one-sample proportion test because you’re comparing your sample proportion (7.5%) to a claimed proportion (5%).
When to Use One-Sample Proportion Test
Use it when:
- ✅ You have ONE sample
- ✅ You’re counting successes/failures (yes/no, pass/fail, click/no-click)
- ✅ You want to compare your sample proportion to a claimed value
- ✅ Sample size is large enough (np ≥ 10 and n(1-p) ≥ 10)
Examples:
- 🎯 Testing if a coin is fair (claimed: 50% heads)
- 📊 Testing if conversion rate meets target (claimed: 10%)
- 🗳️ Testing if candidate has majority support (claimed: 50%)
- 🏥 Testing if cure rate meets standard (claimed: 80%)
The Formula (Don’t Panic!)
The test statistic for proportions is called a Z-score:
\[Z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}\]What each symbol means:
- $\hat{p}$ (p-hat) = Your sample proportion (like 0.075 or 7.5%)
- $p_0$ (p-zero) = The claimed proportion (like 0.05 or 5%)
- $n$ = Sample size (like 200 products)
In plain English: “How far is my sample proportion from the claimed proportion, considering sample size?”
Example: Defective Products
The Story: A factory claims only 5% of products are defective. You inspect 200 products and find 15 defective.
The Data:
- Sample size: n = 200
- Defective found: 15
- Sample proportion: $\hat{p}$ = 15/200 = 0.075 (7.5%)
- Claimed proportion: $p_0$ = 0.05 (5%)
The Question: Is the defect rate higher than claimed?
Step 1: Set Up Your Claims
🔵 Boring Claim (H₀): “The defect rate is 5%”
- In math: p = 0.05
🔴 Interesting Claim (H₁): “The defect rate is HIGHER than 5%”
- In math: p > 0.05
- Note: One-tailed test (we only care if it’s higher)
🎯 How sure do we need to be? α = 0.05 (95% confidence)
Step 2: Check if We Can Use This Test
We need to check if our sample is large enough:
- np₀ = 200 × 0.05 = 10 ✅ (≥ 10)
- n(1-p₀) = 200 × 0.95 = 190 ✅ (≥ 10)
Both conditions met! We can proceed.
Step 3: Calculate the Z-Score
\[Z = \frac{0.075 - 0.05}{\sqrt{\frac{0.05 \times 0.95}{200}}} = \frac{0.025}{\sqrt{0.0002375}} = \frac{0.025}{0.0154} = 1.62\]Breaking it down:
- Top part: 0.075 - 0.05 = 0.025 (2.5% higher than claimed)
- Bottom part: √(0.05 × 0.95 / 200) = 0.0154 (standard error)
- Final Z-score: 0.025 / 0.0154 = 1.62
Step 4: Find the P-Value
For Z = 1.62 (one-tailed, right side):
- P-value ≈ 0.053
Step 5: Make Your Decision
- P-value = 0.053
- Alpha = 0.05
- Since 0.053 > 0.05, we’re just barely outside the “weird zone”
Conclusion (in plain English): “We found 7.5% defective products instead of the claimed 5%. This is higher, but not quite high enough to be statistically significant at the 95% confidence level. It’s borderline - we might want to keep monitoring!”
Technical way to say it: “We fail to reject H₀. There is insufficient evidence at α = 0.05 that the defect rate exceeds 5% (p = 0.053).”
Python Implementation - One-Sample Proportion Test
Let’s code this step by step!
# Step 1: Import the tools we need
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
# Step 2: Enter your data
n = 200 # Sample size (products inspected)
x = 15 # Number of successes (defective products)
p_hat = x / n # Sample proportion
p_0 = 0.05 # Claimed proportion
alpha = 0.05 # Significance level
print("🏭 DEFECTIVE PRODUCTS TEST")
print("="*60)
print(f"Sample size: {n} products")
print(f"Defective found: {x} products")
print(f"Sample proportion: {p_hat:.4f} ({p_hat*100:.2f}%)")
print(f"Claimed proportion: {p_0:.4f} ({p_0*100:.2f}%)")
print("="*60)
# Step 3: Check assumptions
np_0 = n * p_0
n_1_p_0 = n * (1 - p_0)
print(f"\n🔍 CHECKING ASSUMPTIONS:")
print(f"np₀ = {np_0:.1f} (need ≥ 10) {'✅' if np_0 >= 10 else '❌'}")
print(f"n(1-p₀) = {n_1_p_0:.1f} (need ≥ 10) {'✅' if n_1_p_0 >= 10 else '❌'}")
if np_0 >= 10 and n_1_p_0 >= 10:
print("✅ Sample size is large enough! We can proceed.")
else:
print("❌ Sample size too small! Results may not be reliable.")
# Step 4: Calculate Z-score
se = np.sqrt(p_0 * (1 - p_0) / n) # Standard error
z_score = (p_hat - p_0) / se
print(f"\n📊 TEST RESULTS:")
print(f"Standard error: {se:.4f}")
print(f"Z-score: {z_score:.4f}")
# Step 5: Calculate p-value (one-tailed, right side)
p_value = 1 - stats.norm.cdf(z_score)
print(f"P-value (one-tailed): {p_value:.4f}")
# Step 6: Find critical value
z_critical = stats.norm.ppf(1 - alpha)
print(f"Critical value: {z_critical:.4f}")
# Step 7: Make a decision
print("\n🎯 DECISION:")
if p_value < alpha:
print(f"✅ REJECT H₀: Defect rate IS higher than {p_0*100:.0f}%!")
print(f" (p-value {p_value:.4f} < {alpha})")
print(f" We found {p_hat*100:.1f}% defective, significantly higher than claimed")
else:
print(f"❌ FAIL TO REJECT H₀: Not enough evidence")
print(f" (p-value {p_value:.4f} > {alpha})")
print(f" The {p_hat*100:.1f}% defect rate could be due to random variation")
# Step 8: Visualize
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# Plot 1: Bar chart comparing proportions
axes[0].bar(['Claimed', 'Observed'], [p_0, p_hat], color=['blue', 'red'], alpha=0.7)
axes[0].axhline(p_0, color='blue', linestyle='--', linewidth=2, label='Claimed rate')
axes[0].set_ylabel('Proportion Defective')
axes[0].set_title('Defect Rate Comparison')
axes[0].legend()
axes[0].grid(True, alpha=0.3, axis='y')
# Add percentage labels on bars
for i, (label, value) in enumerate([('Claimed', p_0), ('Observed', p_hat)]):
axes[0].text(i, value + 0.005, f'{value*100:.1f}%', ha='center', fontweight='bold')
# Plot 2: Z-score distribution
x = np.linspace(-4, 4, 1000)
y = stats.norm.pdf(x)
axes[1].plot(x, y, 'b-', linewidth=2, label='Standard Normal')
axes[1].axvline(z_score, color='r', linestyle='--', linewidth=2,
label=f'Our Z-score = {z_score:.2f}')
axes[1].axvline(z_critical, color='g', linestyle='--', linewidth=1.5,
label=f'Critical value = {z_critical:.2f}')
# Shade rejection region
x_reject = np.linspace(z_critical, 4, 100)
axes[1].fill_between(x_reject, stats.norm.pdf(x_reject), alpha=0.3, color='red',
label='Rejection Region')
axes[1].set_xlabel('Z-score')
axes[1].set_ylabel('Probability Density')
axes[1].set_title('One-Sample Proportion Test (One-Tailed)')
axes[1].legend()
axes[1].grid(True, alpha=0.3)
plt.tight_layout()
plt.show()
print("\n📊 The graph shows our Z-score is just below the critical value!")
Using statsmodels for Proportion Test
Python has a built-in function that makes this even easier!
from statsmodels.stats.proportion import proportions_ztest
# Your data
count = 15 # Number of defective products
nobs = 200 # Total products inspected
value = 0.05 # Claimed proportion
# Perform one-sample proportion test
z_stat, p_value = proportions_ztest(count, nobs, value, alternative='larger')
print("🏭 QUICK PROPORTION TEST")
print("="*60)
print(f"Defective: {count} out of {nobs} ({count/nobs*100:.1f}%)")
print(f"Claimed: {value*100:.0f}%")
print(f"\nZ-statistic: {z_stat:.4f}")
print(f"P-value: {p_value:.4f}")
if p_value < 0.05:
print("\n✅ Significant! Defect rate is higher than claimed.")
else:
print("\n❌ Not significant. Could be random variation.")
print("="*60)
3. Two-Sample Proportion Test
What is a Two-Sample Proportion Test? (Simple Explanation)
Imagine you’re running an A/B test for a website:
- Version A (Control): 50 out of 500 visitors clicked the button (10%)
- Version B (New design): 80 out of 500 visitors clicked the button (16%)
The Question: Is Version B really better, or could this difference be just random luck?
This is a two-sample proportion test because you’re comparing two proportions to each other.
When to Use Two-Sample Proportion Test
Use it when:
- ✅ You have TWO independent groups
- ✅ Each group has successes/failures (yes/no, click/no-click)
- ✅ You want to compare the two proportions
- ✅ Sample sizes are large enough
Examples:
- 🎯 A/B testing: Version A vs Version B click rates
- 👨👩 Gender differences: Male vs Female success rates
- 💊 Drug comparison: Drug A vs Drug B cure rates
- 🗳️ Political polls: Support for Candidate A in State 1 vs State 2
The Formula (Don’t Panic!)
\[Z = \frac{\hat{p}_1 - \hat{p}_2}{\sqrt{\hat{p}(1-\hat{p})(\frac{1}{n_1} + \frac{1}{n_2})}}\]Where $\hat{p}$ is the pooled proportion:
\[\hat{p} = \frac{x_1 + x_2}{n_1 + n_2}\]What each symbol means:
- $\hat{p}_1$ = Proportion in Group 1 (like 10% click rate)
- $\hat{p}_2$ = Proportion in Group 2 (like 16% click rate)
- $\hat{p}$ = Combined proportion from both groups
- $n_1$, $n_2$ = Sample sizes of each group
In plain English: “How different are the two proportions, considering both sample sizes?”
Example: A/B Testing Website Design
The Story: You’re testing two website designs to see which gets more clicks:
- Version A: 50 clicks out of 500 visitors (10%)
- Version B: 80 clicks out of 500 visitors (16%)
The Question: Is Version B significantly better?
Step 1: Set Up Your Claims
🔵 Boring Claim (H₀): “Both versions have the same click rate”
- In math: p₁ = p₂ (or p₁ - p₂ = 0)
🔴 Interesting Claim (H₁): “The versions have different click rates”
- In math: p₁ ≠ p₂
- Note: Two-tailed test (we’re open to either being better)
🎯 How sure do we need to be? α = 0.05 (95% confidence)
Step 2: Calculate the Proportions
- Version A: $\hat{p}_1$ = 50/500 = 0.10 (10%)
- Version B: $\hat{p}_2$ = 80/500 = 0.16 (16%)
- Pooled: $\hat{p}$ = (50+80)/(500+500) = 130/1000 = 0.13 (13%)
Step 3: Calculate the Z-Score
\[Z = \frac{0.10 - 0.16}{\sqrt{0.13 \times 0.87 \times (\frac{1}{500} + \frac{1}{500})}} = \frac{-0.06}{\sqrt{0.1131 \times 0.004}} = \frac{-0.06}{0.0213} = -2.82\]Step 4: Find the P-Value
For Z = -2.82 (two-tailed):
- P-value ≈ 0.005
Step 5: Make Your Decision
- P-value = 0.005
- Alpha = 0.05
- Since 0.005 < 0.05, we’re in the “weird zone”!
Conclusion (in plain English): “Version B got 16% clicks compared to Version A’s 10%. That’s a 6 percentage point difference, and it’s too big to be just random luck. Version B is genuinely better!”
Technical way to say it: “We reject H₀. There is significant evidence that the two versions have different click rates (p = 0.005).”
Python Implementation - Two-Sample Proportion Test
Let’s code this!
# Step 1: Import tools
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
# Step 2: Enter your data
# Version A
n1 = 500 # Visitors to Version A
x1 = 50 # Clicks on Version A
p1_hat = x1 / n1 # Click rate for Version A
# Version B
n2 = 500 # Visitors to Version B
x2 = 80 # Clicks on Version B
p2_hat = x2 / n2 # Click rate for Version B
alpha = 0.05 # Significance level
print("🖱️ A/B TESTING: WEBSITE CLICK RATES")
print("="*60)
print(f"Version A:")
print(f" Visitors: {n1}")
print(f" Clicks: {x1}")
print(f" Click rate: {p1_hat:.4f} ({p1_hat*100:.1f}%)")
print(f"\nVersion B:")
print(f" Visitors: {n2}")
print(f" Clicks: {x2}")
print(f" Click rate: {p2_hat:.4f} ({p2_hat*100:.1f}%)")
print(f"\nDifference: {(p2_hat - p1_hat)*100:.1f} percentage points")
print("="*60)
# Step 3: Calculate pooled proportion
p_pooled = (x1 + x2) / (n1 + n2)
print(f"\n📊 Pooled proportion: {p_pooled:.4f} ({p_pooled*100:.1f}%)")
# Step 4: Calculate standard error
se = np.sqrt(p_pooled * (1 - p_pooled) * (1/n1 + 1/n2))
print(f"Standard error: {se:.4f}")
# Step 5: Calculate Z-score
z_score = (p1_hat - p2_hat) / se
print(f"Z-score: {z_score:.4f}")
# Step 6: Calculate p-value (two-tailed)
p_value = 2 * stats.norm.cdf(z_score) # Two-tailed (z_score is negative)
print(f"P-value (two-tailed): {p_value:.4f}")
# Step 7: Make a decision
print("\n🎯 DECISION:")
if p_value < alpha:
print(f"✅ REJECT H₀: The versions ARE different!")
print(f" (p-value {p_value:.4f} < {alpha})")
if p2_hat > p1_hat:
print(f" Version B is BETTER! ({p2_hat*100:.1f}% vs {p1_hat*100:.1f}%)")
improvement = ((p2_hat - p1_hat) / p1_hat) * 100
print(f" That's a {improvement:.1f}% relative improvement!")
else:
print(f" Version A is BETTER! ({p1_hat*100:.1f}% vs {p2_hat*100:.1f}%)")
else:
print(f"❌ FAIL TO REJECT H₀: No significant difference")
print(f" (p-value {p_value:.4f} > {alpha})")
print(f" The difference could be random variation")
# Step 8: Visualize
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# Plot 1: Bar chart comparing proportions
versions = ['Version A', 'Version B']
click_rates = [p1_hat, p2_hat]
colors = ['blue', 'green']
bars = axes[0].bar(versions, click_rates, color=colors, alpha=0.7, edgecolor='black')
axes[0].set_ylabel('Click Rate')
axes[0].set_title('Click Rate Comparison')
axes[0].grid(True, alpha=0.3, axis='y')
# Add percentage labels on bars
for bar, rate in zip(bars, click_rates):
height = bar.get_height()
axes[0].text(bar.get_x() + bar.get_width()/2., height + 0.005,
f'{rate*100:.1f}%', ha='center', va='bottom', fontweight='bold')
# Plot 2: Sample sizes and success rates
x_pos = np.arange(2)
successes = [x1, x2]
failures = [n1-x1, n2-x2]
axes[1].bar(x_pos, successes, label='Clicks', color='green', alpha=0.7)
axes[1].bar(x_pos, failures, bottom=successes, label='No Clicks', color='gray', alpha=0.7)
axes[1].set_xticks(x_pos)
axes[1].set_xticklabels(versions)
axes[1].set_ylabel('Number of Visitors')
axes[1].set_title('Clicks vs No Clicks')
axes[1].legend()
axes[1].grid(True, alpha=0.3, axis='y')
plt.tight_layout()
plt.show()
print("\n📊 The graphs show Version B has a higher click rate!")
Using statsmodels for Two-Sample Proportion Test
The easy way:
from statsmodels.stats.proportion import proportions_ztest
# Your data
count = np.array([50, 80]) # Clicks for each version
nobs = np.array([500, 500]) # Visitors for each version
# Perform two-sample proportion test
z_stat, p_value = proportions_ztest(count, nobs)
print("🖱️ QUICK A/B TEST")
print("="*60)
print(f"Version A: {count[0]} clicks out of {nobs[0]} ({count[0]/nobs[0]*100:.1f}%)")
print(f"Version B: {count[1]} clicks out of {nobs[1]} ({count[1]/nobs[1]*100:.1f}%)")
print(f"\nZ-statistic: {z_stat:.4f}")
print(f"P-value: {p_value:.4f}")
if p_value < 0.05:
print("\n✅ Significant difference! The versions perform differently.")
if count[1]/nobs[1] > count[0]/nobs[0]:
print(" Version B is the winner! 🎉")
else:
print(" Version A is the winner! 🎉")
else:
print("\n❌ No significant difference. Keep testing or stick with current version.")
print("="*60)
4. Putting It All Together - Real Examples!
Example 1: Coin Fairness Test (One-Sample)
The Situation: You suspect a coin might be biased. You flip it 100 times and get 60 heads.
import numpy as np
from scipy import stats
from statsmodels.stats.proportion import proportions_ztest
import matplotlib.pyplot as plt
# Data
n = 100 # Number of flips
heads = 60 # Number of heads
p_hat = heads / n # Sample proportion
p_0 = 0.5 # Fair coin proportion
print("🪙 COIN FAIRNESS TEST")
print("="*60)
print(f"Flips: {n}")
print(f"Heads: {heads} ({p_hat*100:.0f}%)")
print(f"Expected (fair coin): {p_0*100:.0f}%")
print("="*60)
# Perform test
z_stat, p_value = proportions_ztest(heads, n, p_0)
print(f"\n📊 TEST RESULTS:")
print(f"Z-statistic: {z_stat:.4f}")
print(f"P-value (two-tailed): {p_value:.4f}")
print("\n🎯 DECISION:")
if p_value < 0.05:
print(f"✅ REJECT H₀: The coin IS biased!")
print(f" (p-value {p_value:.4f} < 0.05)")
if p_hat > 0.5:
print(f" It favors HEADS ({p_hat*100:.0f}%)")
else:
print(f" It favors TAILS ({(1-p_hat)*100:.0f}%)")
else:
print(f"❌ FAIL TO REJECT H₀: The coin appears fair")
print(f" (p-value {p_value:.4f} > 0.05)")
print(f" The {p_hat*100:.0f}% heads could be random variation")
# Visualize
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# Plot 1: Pie chart of results
axes[0].pie([heads, n-heads], labels=['Heads', 'Tails'], autopct='%1.0f%%',
colors=['gold', 'silver'], startangle=90)
axes[0].set_title(f'Coin Flip Results (n={n})')
# Plot 2: Bar chart comparing to fair coin
axes[1].bar(['Expected\n(Fair Coin)', 'Observed'], [0.5, p_hat],
color=['blue', 'red'], alpha=0.7, edgecolor='black')
axes[1].axhline(0.5, color='blue', linestyle='--', linewidth=2, label='Fair coin (50%)')
axes[1].set_ylabel('Proportion of Heads')
axes[1].set_title('Expected vs Observed')
axes[1].legend()
axes[1].grid(True, alpha=0.3, axis='y')
plt.tight_layout()
plt.show()
Example 2: Gender Differences in Product Preference (Two-Sample)
The Situation: A company wants to know if men and women prefer their product at different rates.
- Men: 120 out of 400 prefer it (30%)
- Women: 160 out of 400 prefer it (40%)
import numpy as np
from scipy import stats
from statsmodels.stats.proportion import proportions_ztest
import matplotlib.pyplot as plt
# Data
n_men = 400
prefer_men = 120
p_men = prefer_men / n_men
n_women = 400
prefer_women = 160
p_women = prefer_women / n_women
print("👨👩 GENDER PREFERENCE ANALYSIS")
print("="*60)
print(f"Men:")
print(f" Sample size: {n_men}")
print(f" Prefer product: {prefer_men} ({p_men*100:.1f}%)")
print(f"\nWomen:")
print(f" Sample size: {n_women}")
print(f" Prefer product: {prefer_women} ({p_women*100:.1f}%)")
print(f"\nDifference: {(p_women - p_men)*100:.1f} percentage points")
print("="*60)
# Perform test
count = np.array([prefer_men, prefer_women])
nobs = np.array([n_men, n_women])
z_stat, p_value = proportions_ztest(count, nobs)
print(f"\n📊 TEST RESULTS:")
print(f"Z-statistic: {z_stat:.4f}")
print(f"P-value (two-tailed): {p_value:.4f}")
print("\n🎯 DECISION:")
if p_value < 0.05:
print(f"✅ REJECT H₀: There IS a gender difference!")
print(f" (p-value {p_value:.4f} < 0.05)")
if p_women > p_men:
print(f" Women prefer it MORE ({p_women*100:.1f}% vs {p_men*100:.1f}%)")
else:
print(f" Men prefer it MORE ({p_men*100:.1f}% vs {p_women*100:.1f}%)")
else:
print(f"❌ FAIL TO REJECT H₀: No significant gender difference")
print(f" (p-value {p_value:.4f} > 0.05)")
# Visualize
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# Plot 1: Grouped bar chart
x = np.arange(2)
width = 0.35
prefer = [prefer_men, prefer_women]
not_prefer = [n_men - prefer_men, n_women - prefer_women]
axes[0].bar(x - width/2, prefer, width, label='Prefer', color='green', alpha=0.7)
axes[0].bar(x + width/2, not_prefer, width, label='Don\'t Prefer', color='red', alpha=0.7)
axes[0].set_xticks(x)
axes[0].set_xticklabels(['Men', 'Women'])
axes[0].set_ylabel('Number of People')
axes[0].set_title('Product Preference by Gender')
axes[0].legend()
axes[0].grid(True, alpha=0.3, axis='y')
# Plot 2: Proportion comparison
axes[1].bar(['Men', 'Women'], [p_men, p_women], color=['blue', 'pink'],
alpha=0.7, edgecolor='black')
axes[1].set_ylabel('Proportion Who Prefer Product')
axes[1].set_title('Preference Rate Comparison')
axes[1].grid(True, alpha=0.3, axis='y')
# Add percentage labels
for i, (label, value) in enumerate([('Men', p_men), ('Women', p_women)]):
axes[1].text(i, value + 0.01, f'{value*100:.1f}%', ha='center', fontweight='bold')
plt.tight_layout()
plt.show()
Example 3: Before/After Marketing Campaign (Two-Sample)
The Situation: A company runs a marketing campaign and wants to see if conversion rates improved.
- Before campaign: 80 out of 1000 visitors converted (8%)
- After campaign: 120 out of 1000 visitors converted (12%)
import numpy as np
from statsmodels.stats.proportion import proportions_ztest
import matplotlib.pyplot as plt
# Data
n_before = 1000
convert_before = 80
p_before = convert_before / n_before
n_after = 1000
convert_after = 120
p_after = convert_after / n_after
print("📈 MARKETING CAMPAIGN EFFECTIVENESS")
print("="*60)
print(f"Before Campaign:")
print(f" Visitors: {n_before}")
print(f" Conversions: {convert_before} ({p_before*100:.1f}%)")
print(f"\nAfter Campaign:")
print(f" Visitors: {n_after}")
print(f" Conversions: {convert_after} ({p_after*100:.1f}%)")
print(f"\nImprovement: {(p_after - p_before)*100:.1f} percentage points")
# Calculate relative improvement
relative_improvement = ((p_after - p_before) / p_before) * 100
print(f"Relative improvement: {relative_improvement:.1f}%")
print("="*60)
# Perform test (one-tailed: we only care if it improved)
count = np.array([convert_before, convert_after])
nobs = np.array([n_before, n_after])
z_stat, p_value_two = proportions_ztest(count, nobs)
p_value_one = p_value_two / 2 # One-tailed
print(f"\n📊 TEST RESULTS:")
print(f"Z-statistic: {z_stat:.4f}")
print(f"P-value (one-tailed): {p_value_one:.4f}")
print("\n🎯 DECISION:")
if p_value_one < 0.05 and z_stat < 0: # z_stat negative means after > before
print(f"✅ REJECT H₀: Campaign WORKED!")
print(f" (p-value {p_value_one:.4f} < 0.05)")
print(f" Conversion rate improved from {p_before*100:.1f}% to {p_after*100:.1f}%")
print(f" That's a {relative_improvement:.1f}% relative improvement! 🎉")
else:
print(f"❌ FAIL TO REJECT H₀: No significant improvement")
print(f" (p-value {p_value_one:.4f} > 0.05)")
# Visualize
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# Plot 1: Conversion rates
axes[0].bar(['Before', 'After'], [p_before, p_after],
color=['red', 'green'], alpha=0.7, edgecolor='black')
axes[0].set_ylabel('Conversion Rate')
axes[0].set_title('Conversion Rate: Before vs After Campaign')
axes[0].grid(True, alpha=0.3, axis='y')
# Add percentage labels and arrow
for i, (label, value) in enumerate([('Before', p_before), ('After', p_after)]):
axes[0].text(i, value + 0.005, f'{value*100:.1f}%', ha='center', fontweight='bold')
# Add improvement arrow
axes[0].annotate('', xy=(1, p_after), xytext=(0, p_before),
arrowprops=dict(arrowstyle='->', lw=2, color='blue'))
axes[0].text(0.5, (p_before + p_after)/2, f'+{(p_after-p_before)*100:.1f}pp',
ha='center', fontweight='bold', color='blue', fontsize=12)
# Plot 2: Stacked bar showing conversions
periods = ['Before', 'After']
conversions = [convert_before, convert_after]
non_conversions = [n_before - convert_before, n_after - convert_after]
axes[1].bar(periods, conversions, label='Converted', color='green', alpha=0.7)
axes[1].bar(periods, non_conversions, bottom=conversions,
label='Did Not Convert', color='gray', alpha=0.7)
axes[1].set_ylabel('Number of Visitors')
axes[1].set_title('Conversions: Before vs After')
axes[1].legend()
axes[1].grid(True, alpha=0.3, axis='y')
plt.tight_layout()
plt.show()
5. Practice Exercises - Your Turn!
Exercise 1: Dice Fairness (One-Sample - Easy)
The Problem: You roll a die 120 times and get the number 6 exactly 25 times. A fair die should show 6 about 1/6 of the time (16.67%). Is this die fair?
Your Tasks:
- What’s the claimed proportion?
- What’s your sample proportion?
- Set up H₀ and H₁
- Run the test in Python
- What’s your conclusion?
# Write your code here!
# Hint: Claimed proportion = 1/6 ≈ 0.1667
💡 Click to see solution
```python import numpy as np from statsmodels.stats.proportion import proportions_ztest print("🎲 DICE FAIRNESS TEST") print("="*60) # Given information n = 120 # Number of rolls sixes = 25 # Number of times we got 6 p_hat = sixes / n # Sample proportion p_0 = 1/6 # Fair die proportion (16.67%) # Question 1 & 2 print("Q1 & Q2: Proportions") print(f" Claimed proportion (fair die): {p_0:.4f} ({p_0*100:.2f}%)") print(f" Sample proportion: {p_hat:.4f} ({p_hat*100:.2f}%)\n") # Question 3: Hypotheses print("Q3: Set up hypotheses:") print(" H₀: p = 1/6 (die is fair)") print(" H₁: p ≠ 1/6 (die is NOT fair)") print(" α = 0.05 (95% confidence)\n") # Question 4: Run the test z_stat, p_value = proportions_ztest(sixes, n, p_0) print("Q4: Test Results:") print(f" Z-statistic: {z_stat:.4f}") print(f" P-value (two-tailed): {p_value:.4f}\n") # Question 5: Conclusion print("Q5: Conclusion:") if p_value < 0.05: print(f" ✅ REJECT H₀!") print(f" The die IS biased!") print(f" (p-value {p_value:.4f} < 0.05)") if p_hat > p_0: print(f" It shows 6 TOO OFTEN ({p_hat*100:.1f}% vs expected {p_0*100:.1f}%)") else: print(f" It shows 6 TOO RARELY ({p_hat*100:.1f}% vs expected {p_0*100:.1f}%)") else: print(f" ❌ FAIL TO REJECT H₀") print(f" The die appears fair") print(f" (p-value {p_value:.4f} > 0.05)") print(f" Getting {sixes} sixes in {n} rolls is within normal variation") print("\n📊 In Plain English:") print(f" We got 6 on {sixes} out of {n} rolls ({p_hat*100:.1f}%).") print(f" A fair die should show 6 about {p_0*100:.1f}% of the time.") if p_value < 0.05: print(" This difference is too big to be just luck!") print(" The die is probably loaded! 🎲") else: print(" This difference could easily happen by chance.") print(" The die seems fair! ✅") print("="*60) ```Exercise 2: Drug Effectiveness Comparison (Two-Sample - Medium)
The Problem: Two drugs are being tested for effectiveness:
- Drug A: 180 out of 250 patients recovered (72%)
- Drug B: 200 out of 250 patients recovered (80%)
Your Tasks:
- Which test should you use?
- Set up H₀ and H₁
- Run the test in Python
- Calculate the difference in recovery rates
- What’s your conclusion? Which drug is better?
# Write your code here!
💡 Click to see solution
```python import numpy as np from statsmodels.stats.proportion import proportions_ztest import matplotlib.pyplot as plt print("💊 DRUG EFFECTIVENESS COMPARISON") print("="*60) # Given information n_a = 250 recovered_a = 180 p_a = recovered_a / n_a n_b = 250 recovered_b = 200 p_b = recovered_b / n_b # Question 1: Which test? print("Q1: Which test should we use?") print(" • Two different groups (Drug A patients vs Drug B patients)") print(" ➡️ Answer: TWO-SAMPLE PROPORTION TEST\n") # Question 2: Hypotheses print("Q2: Set up hypotheses:") print(" H₀: p_A = p_B (both drugs equally effective)") print(" H₁: p_A ≠ p_B (drugs have different effectiveness)") print(" α = 0.05 (95% confidence)\n") # Question 3: Run the test count = np.array([recovered_a, recovered_b]) nobs = np.array([n_a, n_b]) z_stat, p_value = proportions_ztest(count, nobs) print("Q3: Test Results:") print(f" Drug A: {recovered_a}/{n_a} recovered ({p_a*100:.1f}%)") print(f" Drug B: {recovered_b}/{n_b} recovered ({p_b*100:.1f}%)") print(f" Z-statistic: {z_stat:.4f}") print(f" P-value (two-tailed): {p_value:.4f}\n") # Question 4: Difference difference = (p_b - p_a) * 100 print("Q4: Difference in recovery rates:") print(f" Drug B - Drug A = {difference:.1f} percentage points") print(f" Relative improvement: {(difference/p_a/100)*100:.1f}%\n") # Question 5: Conclusion print("Q5: Conclusion:") if p_value < 0.05: print(f" ✅ REJECT H₀!") print(f" The drugs ARE significantly different!") print(f" (p-value {p_value:.4f} < 0.05)") if p_b > p_a: print(f" Drug B is BETTER! ({p_b*100:.1f}% vs {p_a*100:.1f}%)") print(f" Drug B has {difference:.1f} percentage points higher recovery rate") else: print(f" Drug A is BETTER! ({p_a*100:.1f}% vs {p_b*100:.1f}%)") else: print(f" ❌ FAIL TO REJECT H₀") print(f" No significant difference found") print(f" (p-value {p_value:.4f} > 0.05)") print(f" Both drugs appear equally effective") print("\n📊 In Plain English:") print(f" Drug A helped {p_a*100:.0f}% of patients recover.") print(f" Drug B helped {p_b*100:.0f}% of patients recover.") if p_value < 0.05: print(f" This {difference:.0f} percentage point difference is real!") print(f" Drug B is the clear winner! 💊✨") else: print(f" The {difference:.0f} percentage point difference could be random.") print(f" Both drugs work about the same.") print("="*60) # Visualize plt.figure(figsize=(12, 5)) # Plot 1: Recovery rates plt.subplot(1, 2, 1) plt.bar(['Drug A', 'Drug B'], [p_a, p_b], color=['blue', 'green'], alpha=0.7, edgecolor='black') plt.ylabel('Recovery Rate') plt.title('Drug Effectiveness Comparison') plt.grid(True, alpha=0.3, axis='y') # Add percentage labels for i, (drug, rate) in enumerate([('Drug A', p_a), ('Drug B', p_b)]): plt.text(i, rate + 0.02, f'{rate*100:.1f}%', ha='center', fontweight='bold') # Plot 2: Stacked bars plt.subplot(1, 2, 2) drugs = ['Drug A', 'Drug B'] recovered = [recovered_a, recovered_b] not_recovered = [n_a - recovered_a, n_b - recovered_b] plt.bar(drugs, recovered, label='Recovered', color='green', alpha=0.7) plt.bar(drugs, not_recovered, bottom=recovered, label='Not Recovered', color='red', alpha=0.7) plt.ylabel('Number of Patients') plt.title('Recovery Outcomes') plt.legend() plt.grid(True, alpha=0.3, axis='y') plt.tight_layout() plt.show() ```Exercise 3: Your Own Example! (Challenge)
Create your own proportion test scenario!
Option A: One-Sample Proportion Test
import numpy as np
from statsmodels.stats.proportion import proportions_ztest
# Example: Test if success rate meets target
print("MY ONE-SAMPLE PROPORTION TEST")
print("="*60)
print("Scenario: [Describe your situation]")
print("Claimed proportion: [e.g., 50%]")
print("="*60)
# Your data
# n = ... # Sample size
# successes = ... # Number of successes
# p_0 = ... # Claimed proportion
# Run test
# z_stat, p_value = proportions_ztest(successes, n, p_0)
# Make conclusion
Option B: Two-Sample Proportion Test
import numpy as np
from statsmodels.stats.proportion import proportions_ztest
# Example: Compare two groups
print("MY TWO-SAMPLE PROPORTION TEST")
print("="*60)
print("Scenario: [Describe your comparison]")
print("="*60)
# Your data
# count = np.array([successes_group1, successes_group2])
# nobs = np.array([n_group1, n_group2])
# Run test
# z_stat, p_value = proportions_ztest(count, nobs)
# Make conclusion
Ideas for scenarios:
- 🎯 Click-through rates: Email A vs Email B
- 🎮 Win rates: Player 1 vs Player 2
- 🏥 Side effect rates: Old drug vs New drug
- 🗳️ Poll results: Support in State A vs State B
- 📱 App crash rates: iOS vs Android
- 🎓 Pass rates: Online course vs In-person course
Summary - What You Learned! 🎓
Key Concepts
1. What Are Proportions?
- Proportions are percentages expressed as decimals (0 to 1)
- Used for yes/no, success/failure, click/no-click data
- Examples: 60% = 0.60, 25% = 0.25
2. One-Sample Proportion Test
- Compare ONE sample proportion to a claimed value
- Example: Is the coin fair? (claimed: 50%)
- Formula: $Z = \frac{\hat{p} - p_0}{\sqrt{\frac{p_0(1-p_0)}{n}}}$
- Python:
proportions_ztest(count, nobs, value)
3. Two-Sample Proportion Test
- Compare TWO sample proportions to each other
- Example: Do men and women have different click rates?
- Uses pooled proportion for standard error
- Python:
proportions_ztest(count_array, nobs_array)
4. Sample Size Requirements
- Need: np ≥ 10 AND n(1-p) ≥ 10
- This ensures the normal approximation works
- If not met, results may be unreliable
Quick Decision Tree
What are you comparing?
│
├─ One proportion to a claimed value?
│ └─ Use ONE-SAMPLE PROPORTION TEST
│ proportions_ztest(count, nobs, value)
│
└─ Two proportions to each other?
└─ Use TWO-SAMPLE PROPORTION TEST
proportions_ztest(count_array, nobs_array)
Python Cheat Sheet
from statsmodels.stats.proportion import proportions_ztest
import numpy as np
# One-Sample Proportion Test
count = 60 # Number of successes
nobs = 100 # Sample size
value = 0.5 # Claimed proportion
z_stat, p_value = proportions_ztest(count, nobs, value)
# Two-Sample Proportion Test
count = np.array([50, 80]) # Successes in each group
nobs = np.array([500, 500]) # Sample sizes
z_stat, p_value = proportions_ztest(count, nobs)
# One-tailed test (add alternative parameter)
z_stat, p_value = proportions_ztest(count, nobs, value,
alternative='larger') # or 'smaller'
# Decision
if p_value < 0.05:
print("Significant difference found!")
else:
print("No significant difference")
Common Mistakes to Avoid ⚠️
- Using proportion tests for averages → Use t-tests instead!
- Forgetting to check sample size requirements → Need np ≥ 10 and n(1-p) ≥ 10
- Confusing proportions with counts → Use proportions (0-1), not percentages (0-100)
- Using two-sample test for before/after → If same subjects, consider paired t-test
- Ignoring practical significance → Small p-value doesn’t always mean important difference
Proportion vs Mean Tests
Use Proportion Tests when:
- ✅ Data is yes/no, success/failure, click/no-click
- ✅ You’re comparing percentages
- ✅ Example: “60% of customers are satisfied”
Use Mean Tests (t-tests) when:
- ✅ Data is continuous numbers
- ✅ You’re comparing averages
- ✅ Example: “Average customer satisfaction score is 7.5/10”
Real-World Applications
Marketing & Business:
- A/B testing website designs
- Comparing conversion rates
- Customer satisfaction surveys
- Click-through rate analysis
Healthcare:
- Drug effectiveness (cure rates)
- Side effect rates
- Patient recovery rates
- Treatment success rates
Quality Control:
- Defect rates
- Product failure rates
- Compliance rates
- Error rates
Politics & Social Science:
- Poll results and voter preferences
- Survey response rates
- Demographic comparisons
- Public opinion changes
What’s Next?
Congratulations! You’ve completed the Hypothesis Testing Methods series! 🎉
You now know:
- ✅ Chapter 4.1: Single Sample Tests (Z-test, T-test)
- ✅ Chapter 4.2: Two Sample Tests (Independent, Paired)
- ✅ Chapter 4.3: Proportion Tests (One-sample, Two-sample)
Next topics to explore:
- Chi-Square Tests: Testing relationships between categorical variables
- ANOVA: Comparing more than two groups
- Regression Analysis: Predicting outcomes and finding relationships
- Non-Parametric Tests: When your data doesn’t meet assumptions
Keep practicing! The more you use these tests, the more intuitive they become! 🚀
Remember: Statistics is about making informed decisions with data. Always ask yourself:
- What question am I trying to answer?
- What type of data do I have?
- Which test is appropriate?
- What do my results mean in the real world?
You’re not just calculating p-values - you’re solving real problems! 💡