Hypothesis tests

S2 Topic 4, spec point by spec point (Hypothesis tests). Each spec point has the method, worked examples written the way the mark scheme wants them, how the marks go and the mistakes that cost the most marks.

Spec4.1Populations, censuses and samples

Understand population, census and sample, sampling unit and sampling frame, and the advantages and disadvantages of a census and a sample survey.

Vocabulary

Term Meaning
Population the whole set of items you're interested in
Census data collected from every member of the population
Sample data from some of the population
Sampling unit one individual item that can be sampled (e.g. a patient)
Sampling frame a list of all the sampling units (e.g. a list of all patients)
Advantages Disadvantages
Census completely accurate, unbiased slow, expensive, hard with large populations; impossible if testing destroys the item
Sample quicker, cheaper, less data to process; the only option for destructive testing may not represent the population (natural variation, bias)

frame and unit

A hospital surveys patients' opinions. Sampling unit: a patient. Sampling frame: a list (database) of all the hospital's patients.

  • B1 for a full list of the population ("a list of opinions" or a partial list is wrong) (WST02A Jun 2026 Q3a).
  • Sampling units must be the individuals ("customers", not "customer opinion") (WST02A Oct 2025 Q2b).
  • Why a sample: "quicker", "cheaper", "easier" (WST02 Jan 2021 Q6a); a census when testing destroys the item is pointless.

Often asked as: one-mark definitions and reasons, in context.

Confusing sampling unit (an individual) with sampling frame (the list).

Spec4.2Statistics and sampling distributions

Understand a statistic and its sampling distribution.

The idea

Finding a sampling distribution

  1. List every possible sample (or the types of sample and how many orders each has).
  2. Calculate the statistic for each.
  3. Add up the probabilities for each value.

total score

A large bag has black and white beads in the ratio \(2 : 5\). Three beads are taken; each black scores 4 and each white 1. Find the sampling distribution of the total \(X\).

\(P(B) = \frac27\), \(P(W) = \frac57\). Values: 3 (WWW), 6 (BWW), 9 (BBW), 12 (BBB).

\(P(X = 3) = \left(\frac57\right)^3 = \frac{125}{343}\), \(P(X = 6) = 3\cdot\frac27\left(\frac57\right)^2 = \frac{150}{343}\), \(P(X = 9) = 3\left(\frac27\right)^2\frac57 = \frac{60}{343}\), \(P(X = 12) = \frac{8}{343}\).

  • B1 for the values (no extras), B1 for the probabilities of each type, M1 for each \(p^3\), \(3p^2(1 - p)\), \(3p(1 - p)^2\) form, A1 for the table (WST02 Jun 2025 Q4a).
  • Listing samples: B1 for the distinct sets, B1 for all ordered samples with no extras (WST02A Jun 2026 Q5a).
  • Medians without replacement: the possible medians, then the probability of each (WST02 Jun 2022 Q7c).

Often asked as: very common (59 questions): sampling distributions of the total, mean, median, range or difference for small samples.

Forgetting the number of orders (the factor 3 for "one black, two white").

Including an unknown parameter in a statistic.

Spec4.3The idea of a hypothesis test

Understand and interpret a hypothesis test, including null and alternative hypotheses, and its use in refining models.

The logic

A hypothesis test asks: "If the claim were true, would a result this extreme be surprising?"

Conclusions

Two sentences:

  1. Statistical: "reject \(H_0\)" (significant) or "do not reject \(H_0\)" (not significant).
  2. In context, using the words of the question: "there is evidence that the proportion of peas that germinate has changed."

Never say \(H_0\) is "proved" or "accepted". Not rejecting just means there isn't enough evidence against it.

interpreting a result

A two-tailed test of \(H_0: p = 0.35\) has critical region \(X \leqslant 1\) or \(X \geqslant 10\). 8 of 15 peas germinated.

8 is not in the critical region, so do not reject \(H_0\). There is insufficient evidence that the germination rate differs from 35%: the company's claim is supported.

  • M1 for a non-contextual statement matching the critical region, A1ft for a contextual conclusion (WST02A Jan 2026 Q4b).
  • Hypotheses are in terms of the parameter (\(p\) or \(\lambda\)), never the test statistic or sample values (WST02 Jan 2016 Q5d).

Often asked as: inside every hypothesis test question.

\(H_0: X = 3\) or hypotheses in words only.

"Accept \(H_0\)" or a conclusion with no context.

Spec4.4Critical regions

Find and use the critical region, using a statistic as a test statistic.

The idea

The critical region is the set of values of the test statistic that would make you reject \(H_0\). The boundary value is the critical value. Because the distribution is discrete, the actual significance level (the probability of landing in the critical region if \(H_0\) is true) is usually a bit less than the nominal level.

Finding it with tables

Write the critical region as \(X \leqslant 1\), not "\(P(X \leqslant 1)\)".

two-tailed critical region

\(H_0: p = 0.35\), \(H_1: p \neq 0.35\), \(X \sim B(15, 0.35)\), 5% level. Find the critical region and the actual significance level.

Lower: \(P(X \leqslant 1) = 0.0142 < 0.025\) but \(P(X \leqslant 2) = 0.0617\). So \(X \leqslant 1\).

Upper: \(P(X \geqslant 10) = 0.0124 < 0.025\) but \(P(X \geqslant 9) = 0.0422\). So \(X \geqslant 10\).

CR: \(X \leqslant 1\) or \(X \geqslant 10\). Actual level \(= 0.0142 + 0.0124 = 0.0266\).

sample size so that 0 is significant

The proportion of unfit pears is 0.07. Find the smallest \(n\) for which finding no unfit pears is in the 5% critical region for a decrease.

\(P(Y = 0) = 0.93^n < 0.05 \Rightarrow n > \dfrac{\log 0.05}{\log 0.93} = 41.3\), so \(n = 42\).

  • B1 for hypotheses, M1 for the right distribution, A1 A1 for each tail (stating "\(P(X \leqslant 1)\)" instead of "\(X \leqslant 1\)" loses an A) (WST02A Jan 2026 Q4a).
  • Poisson tails: B1 for \(\text{Po}(10)\), M1 for a correct tail probability, A1 A1 (WST02 Jan 2017 Q5c).
  • Logs: M1 for \(0.93^n\), M1 for the inequality with logs, A1 (WST02 Jun 2022 Q4a).

Often asked as: find a critical region; actual significance level; then use it with an observed value.

Using 5% in each tail of a two-tailed test.

Off-by-one errors in the upper tail (\(P(X \geqslant c) = 1 - P(X \leqslant c - 1)\)).

Spec4.5One-tailed and two-tailed tests

Distinguish one-tailed and two-tailed tests.

Which one?

Read the claim in the question:

Wording \(H_1\) Test
"increased", "more than", "higher" \(p > p_0\) one-tailed (upper)
"decreased", "fewer", "less" \(p < p_0\) one-tailed (lower)
"changed", "different", "not equal to" \(p \neq p_0\) two-tailed

In a two-tailed test, split the significance level equally between the two tails. When using a \(p\)-value, compare it with half the level (or double the tail probability).

choosing the tails

"Test whether the proportion of faulty components has decreased" means \(H_1: p < 0.2\), a one-tailed test using only the lower tail.

"Test whether the mean number of calls has changed" means \(H_1: \lambda \neq 6\), two-tailed.

No questions in the bank test this point on its own: it decides the set-up of every test. The hypotheses mark is often lost by choosing the wrong tail.

Spec4.6Tests for a binomial p and a Poisson mean

Carry out hypothesis tests for the parameter \(p\) of a binomial distribution and for the mean of a Poisson distribution, using tables or a Normal approximation for the binomial.

Method (a complete test)

  1. Hypotheses: \(H_0: p = p_0\) and \(H_1\) (one or two tails), or the same with \(\lambda\).
  2. Distribution under \(H_0\): \(X \sim B(n, p_0)\) or \(\text{Po}(\lambda_0)\) (scaled to the interval).
  3. Either find the probability of a result at least as extreme as the observed value (\(p\)-value), or find the critical region.
  4. Compare with the significance level (half of it per tail for two-tailed).
  5. Conclude statistically and in context.

For large \(n\), use an approximation: Poisson (small \(p\)) or Normal with continuity correction (\(p\) near 0.5).

binomial, lower tail

Previously 20% of components were faulty. After a change of supplier, 2 of 25 are faulty. Test at 10% whether the proportion has decreased.

\(H_0: p = 0.2\), \(H_1: p < 0.2\). Under \(H_0\), \(X \sim B(25, 0.2)\).

\(P(X \leqslant 2) = 0.0982 < 0.10\): significant, reject \(H_0\).

There is evidence that the probability of a faulty component has decreased.

using a Poisson approximation

The proportion with an allergy is thought to be \(\frac{1}{200}\). In a sample of 500, 5 have it. Test at 5% whether the proportion is higher.

\(H_0: p = \frac{1}{200}\), \(H_1: p > \frac{1}{200}\). \(B(500, 0.005) \approx \text{Po}(2.5)\).

\(P(X \geqslant 5) = 1 - P(X \leqslant 4) = 0.1088 > 0.05\): not significant, do not reject \(H_0\).

There is insufficient evidence that the proportion with the allergy is higher than \(\frac{1}{200}\).

  • B1 for both hypotheses in terms of \(p\) or \(\lambda\), M1 for the right probability (or critical region), M1 for a correct non-contextual statement, A1 for a contextual conclusion (WST02A Jun 2026 Q4e).
  • With a Normal approximation: M1 A1 for \(N(np, np(1 - p))\), M1 standardise, M1 continuity correction, A1, A1 conclusion using the word "proportion" (WST02 May 2024 Q3f).
  • Poisson approximation: B1 for \(\text{Po}(np)\) (WST02 Jun 2017 Q4b).

Often asked as: every S2 paper has a full test. Sometimes several parts: critical region, then the actual significance level, then a test with a new sample using an approximation.

Comparing \(P(X = 2)\) instead of \(P(X \leqslant 2)\) with the level.

Using the observed proportion in the hypotheses.

A conclusion that sounds certain ("this proves...").