What Is a P-Value? Worked by Shuffling Nine Real Orders 126 Ways

By Michael Nocito , data analyst · Published August 9, 2026 By the end of this page you can say what a p-value measures in one sentence, compute one by hand with no distribution theory at all, and name the four things people routinely claim a p-value says that it does not. The worked example is nine real orders where two regions differ by 168.50, and the answer comes out of counting rather than out of a table. Here is what to actually do today. Any time you are about to report that two groups differ, write down the two group sizes first. If either is under about twenty, a p-value will almost certainly come back large no matter how real the difference is, and the honest report is the difference, the sizes, and an interval, not a verdict. The short version: a p-value is the share of results at least as extreme as yours that you would get if the thing you are testing had no effect at all. Small means your result would be unusual under nothing-happening. It does not mean the effect is large, and it does not mean the effect is real. That definition is doing a lot of work in one sentence, so it gets the picture. The original carries a diagram here. In words: A histogram built from small dots, one dot per outcome, arranged in ten vertical columns of different heights standing on a horizontal baseline. The columns rise from one dot at the far left to a peak of twenty-seven dots just right of centre, then fall away to a single dot at the far right, giving the whole shape a rounded hump centred slightly left of the middle of the picture. Two vertical dashed lines cut down through the shape, one on the left of the hump and one on the right, placed symmetrically about the hump's centre. The dots lying in the two tails beyond those lines are drawn in a darker, warmer shade, and the dots in the bulk between them are drawn in a lighter blue, so the tails stand out from the middle. In the two columns the dashed lines pass through, the darker dots are stacked at the bottom of the co