If the observed probability is greater than alpha (i.e., greater than .01, .05, .10, or whatever value you select), do not reject the null hypothesis. Tests of statistical significance are used to...

If the observed probability is greater than alpha (i.e., greater than .01, .05, .10, or whatever value you select), do not reject the null hypothesis. Tests of statistical significance are used to make decisions, and with these or any other procedures, including visual analysis, there’s always some chance that the decision will be wrong. The different types of incorrect and correct decisions are illustrated in Figure 6 , and as shown in this figure there are two types of possible decision errors. First, you might conclude that a difference exists when it really doesn’t; that is, you reject the null hypothesis when it shouldn’t be rejected. This is called a Type I error by statisticians. The probability of making a Type I error is determined by your alpha level. Second, you might conclude that a difference doesn’t exist when it really does; that is, you don’t reject the null hypothesis

Figure 3