Skip to content

Multiplicity Issues

Multiplicity Issues:

Multiplicity issues arise in a number of contexts, but they generally boil down to the same thing: repeated looks at a data set in different ways, until something “statistically significant” emerges. See multiple comparisons for how to handle multiple pairwise testing in conjunction with ANOVA. In observational studies, problems arise when many different models are applied to the same data, particularly when a highly specific thesis-to-be-tested is not stated in advance. Stanley Young draws readers attention to a study (Mostofsky et al) in which an association is claimed between certain constituents of air pollution and ischemic stroke (American Journal of Epidemiology, December 27, 2012, letter to the editor). Young points out that, given the number of predictors and adjustors in the data set, 537 million models are possible to construct. How many models were tried before a statistically-significant association was found? Where multiple testing occurs and is properly disclosed, the false discovery rate (the expected number of false “significant” results with a given number of multiple tests under null models) can be used to control for inflated Type I error.

Browse Other Glossary Entries

Test Yourself

Planning on taking an introductory statistics course, but not sure if you need to start at the beginning? Review the course description for each of our introductory statistics courses and estimate which best matches your level, then take the self test for that course. If you get all or almost all the questions correct, move on and take the next test.

Data Analytics

Considering becoming adata scientist, customer analyst or our data science certificate program?

Analytics Quiz

Advanced Statistics Quiz

Statistics Quiz


Looking at statistics for graduate programs or to enhance your foundational knowledge?

Statistics 1 Quiz

Regression Quiz

Regression Quiz


Entering the biostatistics field? Test your skill here.

Biostatistics Quiz

Advanced Statistics Quiz

Statistics 2 Quiz

Stay Informed

Our Blog

Read up on our latest blogs


Learn about our certificate programs


Find the right course for you

Contact Us

We'd love to answer your questions

Our mentors and academic advisors are standing by to help guide you towards the courses or program that makes the most sense for you and your goals.

300 W Main St STE 301, Charlottesville, VA 22903

(434) 973-7673

By submitting your information, you agree to receive email communications from All information submitted is subject to our privacy policy. You may opt out of receiving communications at any time.