Decision Errors

Chapter 14

Hypothesis Testing Framework

  1. State the null hypothesis and the alternative hypothesis
  1. Choose a sample, collect and analyze the data
  1. How likely is it to see data like what we observed, if the null hypothesis were true?
  1. If very unlikely, we reject the null hypothesis. Otherwise, we cannot reject the null claim.

Making a decision

If the conditional probability of obtaining our test statistic, or more extreme, is very small: \[ p < \alpha \] We have evidence to reject the null hypothesis.

This probability is the p-value and the discernment level (\(\alpha\)) (or significance level) is our threshold for “very small”.

Making a decision

On the other hand, if the p-value is greater (or equal to) \(\alpha\), we say the results are not statistically significant and we fail to reject.


Note: we don’t say \(H_0\) is true – just that we don’t have evidence to say it’s not!

Choosing a discernability level

This should always be done before seeing the data!


What are the consequences of making an incorrect decision?

Type I Error

Suppose that the null hypothesis is actually true


  • we might correcly fail to reject (good decision)

  • we might incorrectly reject the null hypothesis

Type II Error

Suppose that the null hypothesis is actually not true


  • we might correcly reject (good decision)

  • we might incorrectly fail to reject the null hypothesis

Decision Errors


Suppose the null hypothesis is \(H_0\)

  • Type I Error: rejecting \(H_0\) when it is actually true
  • Type II Error: not rejecting \(H_0\) when it is actually false

How likely these errors are depends on discernment level.

What are the consequences for making each type of error?

Choose \(\alpha\) accordingly!

Example

A researcher believes that the mean number of pesticides is higher in the Willamette river than compared to a 1996 report that cited 36 different pesticides. The researcher collects samples from the river over a year’s time and found a significant increase in the mean number of pesticides.

  • State the null and alternative hypotheses.
  • What has happened if a type I error occurs – what are the potential consequences?
  • What has happened if a type II error occurs – what are the potential consequences?

Controlling for Errors

  • To avoid Type I errors, make it harder to reject \(H_0\) – make discernment level smaller

  • To avoid Type II errors, make it easier to reject \(H_0\) – make discernment level bigger

For Thursday…

  • Read about one-sided and two-side tests


  • Homework: Section 14.6, #2, 10 – plus any problems from today that weren’t completed (or still have questions about)