The statement being assessed in a test of significance is called the null hypothesis. The test of significance is designed to assess the strength of the evidence against the null hypothesis (Ho). Usually the null hypothesis is a statement of no difference or no effect. The probability of getting an outcome at least as far from what we would expect if Ho were true as was the actually observed outcomes is called the P-value. The smaller the P-value is, the stronger the evidence is against Ho provided by the data.

Because the strength of the evidence provided by the data is measured by the P-value, we need only say how small a P-value we insist on. This decisive value is called the significance level. If it =0.05, we are requiring the data evidence against Ho to be so strong that it would happen no more than 5% of the time (1/20) when Ho is really true. If we make it =0.01, we are insisting on stronger evidence against Ho, evidence so strong that it would appear only 1% of the time (1/100) if Ho is really true.

Steps in a Test of Significance:

1. Choose the null hypothesis (Ho) and the alternative hypothesis (Ha or H1). The test is designed to assess the strength of the evidence against Ho. Ha is a statement of the alternative we will accept if the evidence enables us to reject Ho.

2. Choose the significance level. This states how much evidence against Ho we will accept as decisive.

3. Choose the test statistic on which the test will be based. This is a statistic which measures how well the data conform to Ho.

4. Find the P-value for the observed data. This is the probability that the test statistic would weigh against Ho at least as strongly as it does for these data, if Ho were in fact true. If the P-value is less than or equal to the level of significance, the test was statistically significant at the chosen level of significance.

If we reject Ho when in fact Ho is true, this is a TYPE I error.

If we accept Ho when in fact Ha is true, this is a TYPE II error.