Glossary

Permutation Tests

Permutation Tests:

A permutation test involves the shuffling of observed data to determine how unusual an observed outcome is. A typical problem involves testing the hypothesis that two or more samples might belong to the same population. The permutation test proceeds as follows:

1. Combine the observations from all the samples

2. Shuffle them and and redistribute them it in resamples of the same sizes as the original samples.

3. Record the statistic of interest.

4. Repeat 2-3 many times

5. Determine how often the resampled statistic of interest is as extreme as the observed value of the same statistic.

The above procedure is a Monte Carlo permutation test, also called a sampled or approximate permutation test. A permutation test in which all possible shufflings are systematically used is an exhaustive permutation test.

Browse Other Glossary Entries

Test Yourself

Planning on taking an introductory statistics course, but not sure if you need to start at the beginning? Review the course description for each of our introductory statistics courses and estimate which best matches your level, then take the self test for that course. If you get all or almost all the questions correct, move on and take the next test.