Significance [sɪɡˈnɪf.ə.kəns]: A measure of certainty of a statistical result that is used in hypothesis testing. In other words, the result is unlikely to have occured if the hypothesis you want to disprove were true. To be statistically significant, the result would have to be less likely than a significance level chosen beforehand. That level might be 5 % or 1 %, for example. If that’s the case, you win: You’d reject the hypothesis you wanted to disprove since it’s so unlikely to be true.
Say you want to study if a drug has negative side effects. Just to be sure, you’ll assume the drug does have negative side effects, and only change your mind if you’re absolutely sure it doesn’t. You wouldn’t want to poison anyone on a statistical error, after all. If you were to accept that the drug has no side effects, you’d want to only be wrong, say, one out of 100 times. So, you’d set your significance level to 1 %.
You then study 100 patients and observe nausea in two of them. That might be an indicator for actual side effects of the drug, or they might have just eaten something bad. Through some statistical magic, you can estimate how likely it is that only two out of 100 people show side effects if the drug would actually cause them. That estimate is called the p-value. If that is highly unlikely – less likely than your limit of 1 % – you’d change your mind and write in your report: “The result is statistically significant at the significance level 1 %. The hypothesis ‘the drug does have negative side effects’ is rejected” and approve the drug.
However, just because a result is statistically significant, that does not automatically mean that a study is done well, that the result is relevant or, in fact, even reliable. Always look for the corresponding significance level. The lower it is, the more reliable your result. Anything at 5 % or below is generally considered okay.Back to Dictionary