Guides › Interpreting p-values
Interpreting SP 800-22 p-values: pass, fail and N/A
Every SP 800-22 test turns your bit sequence into a p-value. Reading those numbers correctly is the difference between a real finding and a false alarm.
What a p-value means here
Each test assumes the sequence is random, computes a statistic, and asks how likely a statistic at least that extreme would be for random data. That probability is the p-value.
- p ≥ 0.01: pass. The result is consistent with randomness at the significance level α = 0.01 used throughout SP 800-22.
- p < 0.01: fail. A pattern this strong appears less than 1% of the time in random data.
- A large p-value is not "more random". p = 0.93 is no better than p = 0.21. For random data, p-values are spread evenly between 0 and 1.
Why one failure is expected
Because α = 0.01, a perfect random generator still fails any single test about 1% of the time. Across 15 tests the chance that a random sequence fails at least one is 1 − 0.9915 ≈ 14%. The NIST reference code reports several p-values for some tests (for example one per template or per random-walk state), so it produces even more chance failures.
Judging a generator: proportion and uniformity
SP 800-22 section 4.2 evaluates a generator by testing many sequences (m of them) and checking two things for each test.
1. Proportion of passing sequences
The expected pass rate is p̂ = 1 − α = 0.99. The acceptable range is
p̂ ± 3 · sqrt( p̂ (1 − p̂) / m )For m = 100 sequences that is 0.99 ± 0.0298, so the pass rate should be at least about 0.96: roughly 96 or more of the 100 sequences pass each test.
2. Uniformity of the p-values
The p-values of a test across all sequences are sorted into 10 equal bins between 0 and 1, and a χ² test (9 degrees of freedom) checks whether they are evenly spread. The p-values are considered uniform when this second-level value, P-valueT, is at least 0.0001. NIST recommends at least 55 sequences for this check.
When a test keeps failing
| Failing test | Usual cause |
|---|---|
| Frequency, Block Frequency, Cusum | Bias: more ones than zeros overall or in parts of the stream (unbalanced comparator, missing debiasing) |
| Runs, Longest Run | Bits that are correlated with their neighbours, e.g. sampling a slow noise source too fast |
| DFT (Spectral) | Periodic structure, such as interference from a clock or a repeating buffer |
| Linear Complexity, Matrix Rank | Linear structure, typical of LFSR-based or poorly mixed generators |
| Maurer's Universal, Approximate Entropy, Serial | Repeating patterns or low entropy; the data is compressible |
| Everything fails | Usually an input mistake: text read as binary, a header in the file, or repeated data (check your file) |
N/A is not a failure
An N/A result means the sequence was shorter than the test needs. The Overlapping Template, Random Excursions and Random Excursions Variant tests, for instance, expect about 1,000,000 bits, and the Random Excursions tests also need at least 500 cycles of the random walk. Capture more data rather than treating N/A as a fail (minimum length per test).
See your own p-values. NISTTEST.COM shows every test's p-value and pass/fail on screen for free.
Run the free testNISTTEST.COM is an independent implementation of NIST SP 800-22 Rev. 1a and is not affiliated with or endorsed by the National Institute of Standards and Technology.