What is a p-value?
A p-value is the key output of a statistical test, such as a Two Sample t-test, a Normality Test, or a Chi-Squared test.
How do you interpret a p-value?
When you are completing a statistical test, you will have first constructed a Null Hypothesis and an Alternative Hypothesis.
An example of a Null Hypothesis is:
“There is no difference between the average delivery times our suppliers.”
The Alternative Hypothesis to this example would be:
“There is a difference between the average delivery times of our suppliers.”
P-values can be thought of as probability. They represent the probability of the data (that you are using in the test) occurring if the Null Hypothesis were true.
So, if the p-value is very low (typically anywhere below 0.05), then you have low confidence that your Null Hypothesis is true, and so you reject it, and accept the Alternative Hypothesis.
If the p-value is not below 0.05, then there is not enough evidence to reject the Null Hypothesis, and so you accept it.
Technically speaking, it’s important to clarify that if the p-value is above 0.05, you are not proving that the Null Hypothesis is true, you just don’t have enough evidence to reject it.
So, in the example described above, if the p-value were below 0.05, we would decide that there is a statistically significant difference between the average delivery times between our suppliers, and so it would be worthwhile to investigate what is causing that difference, in order to better understand the process.
It’s important to remember that statistical tests must be completed carefully, with full awareness of the assumptions and limitations of the test, in order to ensure that the resulting p-value is valid.
When should you use them?
Advanced statistical tests (such as hypothesis tests) should generally be used after you have completed some preliminary graphical analysis of the data. They are used to help you decide whether the observations that you have made from your graphs are statistically valid.