Statistical Hypothesis Testing: Errors, Power, and Inference

Statistical Hypothesis Testing

1. Statistical Hypothesis

A statistical hypothesis is an assertion about a characteristic or parameter of a population. It's used to perform analysis and can be either rejected or accepted based on the provided information. There are two types of hypotheses:

• Null Hypothesis (H₀): Represents the status quo or the default assumption.
• Alternative Hypothesis (H₁): Represents the claim or the hypothesis we want to test.

Both H₀ and H₁ can be simple (if the parameter has only one value) or compound (if the parameter can take multiple values).

2. Significance Level (α)

The significance level is the probability of making a Type I error (rejecting H₀ when it's actually true). It represents the level of risk we're willing to accept in our analysis. It's often expressed as a percentage (e.g., 5%).

3. Type I Error

A Type I error occurs when we reject the null hypothesis when it's actually true. The probability of a Type I error is denoted by α.

4. Type II Error (β)

A Type II error occurs when we fail to reject the null hypothesis when it's actually false. The probability of a Type II error is denoted by β.

5. Power of a Contrast (1-β)

The power of a contrast is the probability of correctly rejecting the null hypothesis when it's false. It's calculated as 1-β.

6. Statistic

A statistic is any value calculated from a sample.

7. Acceptance Region

The acceptance region is the range of possible values of the test statistic that lead to the acceptance of the null hypothesis.

8. Critical Region

The critical region is the range of possible values of the test statistic that lead to the rejection of the null hypothesis. It contains the values that are considered unlikely if the null hypothesis were true.

9. Confidence Level

The confidence level is the probability that a confidence interval contains the true population parameter.

10. Statistical Inference

Statistical inference is the process of drawing conclusions about a population based on a sample.

11. Optimal Critical Region (Newman-Pearson Theorem)

The Newman-Pearson theorem helps find an optimal critical region that maximizes the power of the test for a given significance level. It requires the following conditions:

1. H₀ and H₁ must be simple hypotheses.
2. We need a sample of size n.
3. We set a significance level α.
4. L₀ is the maximum likelihood function under H₀.
5. L₁ is the maximum likelihood function under H₁.
6. L₀/L₁ ≤ k, where k is a positive constant.

12. Relationship Between Type I and Type II Errors

1. β is not complementary to α. The complement of α is 1-α.
2. For a given sample size, if α decreases, β increases, and vice versa. The probabilities of Type I and Type II errors are inversely related.
3. α and β are not independent of the sample size. They depend on the effect size, the sample size, and the significance level.

13. Stages of a Hypothesis Test

1. Define the null and alternative hypotheses.
2. Choose a significance level.
3. Calculate the test statistic.
4. Determine whether to reject or accept the null hypothesis based on the test statistic and the critical region.

14. Uniformly Most Powerful Region

A uniformly most powerful (UMP) region is similar to the optimal critical region obtained by the Newman-Pearson theorem but works for any parameter value in the alternative hypothesis, as long as it's one-sided.

15. Likelihood Ratio Test

The likelihood ratio test is used when the Newman-Pearson theorem cannot be applied (e.g., when H₀ and H₁ are compound hypotheses). It has the following characteristics:

1. It's a general procedure.
2. It agrees with the Newman-Pearson theorem in the case of simple hypotheses.
3. It doesn't guarantee obtaining the optimal test.
4. It has good properties in large samples (n ≥ 30).
5. It's based on the ratio of likelihood functions.