Skip to content

Explore Courses | Elder Research | Contact | LMS Login

Statistics.com: Data Science, Analytics & Statistics Courses
  • Curriculum
    • Curriculum
    • About Us
    • Testimonials
    • Management Team
    • Faculty Search
    • Teach With Us
    • Credit & Credentialing
  • Courses
    • Explore Courses
    • Course Calendar
    • About Our Courses
    • Course Tour
    • Test Yourself!
  • Mastery Series
    • Mastery Series Program
    • Bayesian Statistics
    • Business Analytics
    • Healthcare Analytics
    • Marketing Analytics
    • Operations Research
    • Predictive Analytics
    • Python for Analytics
    • R Programming
    • Rasch & IRT
    • Spatial Statistics
    • Statistical Modeling
    • Survey Statistics
    • Text Mining and Analytics
  • Certificates
    • Certificate Program
    • Analytics for Data Science
    • Biostatistics
    • Programming for Data Science – R (Novice)
    • Programming for Data Science – R (Experienced)
    • Programming for Data Science – Python (Novice)
    • Programming for Data Science – Python (Experienced)
    • Social Science
  • Degrees
    • Degree Programs
    • Computational Data Analytics Certificate of Graduate Study from Rowan University
    • Health Data Management Certificate of Graduate Study from Rowan University
    • Data Science Analytics Master’s Degree from Thomas Edison State University (TESU)
    • Data Science Analytics Bachelor’s Degree – TESU
    • Mathematics with Predictive Modeling Emphasis BS from Bellevue University
  • Enterprise
    • Organizations
    • Higher Education
  • Resources
    • Blog
    • FAQs & Knowledge Base
    • Glossary
    • Site Map
    • Statistical Symbols
    • Weekly Brief Newsletter Signup
    • Word of the Week
Menu Close
  • Curriculum
    • Curriculum
    • About Us
    • Testimonials
    • Management Team
    • Faculty Search
    • Teach With Us
    • Credit & Credentialing
  • Courses
    • Explore Courses
    • Course Calendar
    • About Our Courses
    • Course Tour
    • Test Yourself!
  • Mastery Series
    • Mastery Series Program
    • Bayesian Statistics
    • Business Analytics
    • Healthcare Analytics
    • Marketing Analytics
    • Operations Research
    • Predictive Analytics
    • Python for Analytics
    • R Programming
    • Rasch & IRT
    • Spatial Statistics
    • Statistical Modeling
    • Survey Statistics
    • Text Mining and Analytics
  • Certificates
    • Certificate Program
    • Analytics for Data Science
    • Biostatistics
    • Programming for Data Science – R (Novice)
    • Programming for Data Science – R (Experienced)
    • Programming for Data Science – Python (Novice)
    • Programming for Data Science – Python (Experienced)
    • Social Science
  • Degrees
    • Degree Programs
    • Computational Data Analytics Certificate of Graduate Study from Rowan University
    • Health Data Management Certificate of Graduate Study from Rowan University
    • Data Science Analytics Master’s Degree from Thomas Edison State University (TESU)
    • Data Science Analytics Bachelor’s Degree – TESU
    • Mathematics with Predictive Modeling Emphasis BS from Bellevue University
  • Enterprise
    • Organizations
    • Higher Education
  • Resources
    • Blog
    • FAQs & Knowledge Base
    • Glossary
    • Site Map
    • Statistical Symbols
    • Weekly Brief Newsletter Signup
    • Word of the Week

Blog

Famous Errors in Statistics

  • September 17, 2020
  • , 5:04 pm
“A little knowledge is a dangerous thing,” said Alexander Pope in 1711; he could have been speaking of the use of statistics by experts in all fields. In this article, we look at three consequential mistakes in the field of statistics. Two of them are famous, the third required a deep dive into the corporate annual reports of

Famous Mistakes in Statistics

“A little knowledge is a dangerous thing,” said Alexander Pope in 1711; he could have been speaking of the use of statistics by experts in all fields. In this article, we look at three consequential mistakes in the field of statistics. Two of them are famous, the third required a deep dive into the corporate annual reports of a U.S. software company.

1. The Truth About Sex

Albert Kinsey’s Sexual Behavior in the Human Male was published in 1948 and, though it was a relatively dry, 800-page scientific work chock full of statistics and tables, it quickly made the national best-seller list. Among its stunning (at the time) findings was that 10% of adult men were homosexuals, and an additional 11% were bisexual. 37% had had at least one homosexual experience. The book generated considerable controversy, and was followed several years later by a similar report on women.

Kinsey’s work was challenged by statisticians and is a good example of a case where big data are not necessarily better data. Kinsey interviewed roughly 5,300 men and 6,000 women in his research, but he followed no clear sampling methodology. His 10% estimate for male homosexuals became widely reported and used, but it turned out to be a wild over-estimate. Among the cohort that would have been adults at the time of Kinsey’s research, recent data show that the number self-identifying as LGBT has been consistently less than 2%.

Where did Kinsey go wrong?

Most likely, the problem lay in his sampling procedures. He did not lay out any definitive sampling methodology and seemed to have proceeded by convenience and by tapping into groups that were accessible (e.g. his students, and prison populations). Gathering information from readily available sources is a fine technique for getting started on a study, to identify possible questions to ask and hypotheses to probe. But convenience sampling, as this is known, is not a good way to make a sensitive quantitative estimate. It is a process prone to bias and inaccuracy. Investigator perspective guides who is surveyed and thus becomes part of the data. The chain process by which subjects refer other subjects over-weights some of the data. Perhaps more important is “volunteer bias,” the bias that occurs when only those who volunteer their information are included. It can be especially strong when dealing with highly sensitive or personal subjects.

The controversy aroused by the Kinsey report brought the American Statistical Association into the picture: a committee of distinguished statisticians (F. Mosteller, G. W. Cochran, J. Tukey, and W. Jenkins) issued a report that concluded:

“…many of the most interesting and provocative statements in the book are not based on the data presented therein, and it is not made clear to the reader on what evidence the statements are based. … Taken cumulatively, these objections amount to saying that much of the writing in the book falls below the level of good scientific writing.”

Moral of the story:

Collecting good data in an unbiased fashion is more important than collecting a lot of data.

2. Almost Bankrupting the Company…

In 1993, Mathsoft Corp., the developer of Mathcad mathematical software, acquired StatSci, the developer of S-PLUS statistical software. Mathcad was an affordable tool popular with engineers: prices were in the hundreds of dollars and the number of users was in the hundreds of thousands. S-PLUS was a high-end graphical and statistical tool used primarily by statisticians: prices were in the thousands of dollars and the number of users was in the thousands.

In looking to boost revenues, Mathsoft turned to an established marketing principle: cross-selling. In other words, try to convince the people who bought product A to buy product B. With the acquisition of a highly-regarded niche product, S-PLUS, and an existing large customer base for Mathcad, Mathsoft decided that the logical thing to do would be to ramp up S-PLUS sales via direct mail to its installed Mathcad user base. It also decided to purchase lists of similar prospective customers for both Mathcad and S-PLUS.

This major mailing program was a bust. It boosted revenues, but it boosted expenses even more, to the point where they threatened the company’s future. The company lost over $13 million in 1993 and 1994 combined—significant numbers for a company that had only $11 million in 1992 revenue.

What happened?

In retrospect, it was clear that the mailings were not well targeted. The costs of the unopened mail exceeded the revenue from the few recipients who did respond. In particular, Mathcad users turned out not to be likely users of S-PLUS. The huge losses could have been avoided through the use of two common statistical techniques:

  1. Doing a test mailing to the various lists being considered to (a) determine whether the list is productive and (b) test different headlines, copy, pricing, etc., to see what works best.
  2. Using predictive modeling techniques to identify which names on a list are most likely to turn into customers.

Moral of the story:

Even a company built on statistical algorithms is prone to non-statistical thinking.

Epilogue:

In 2008, Tibco, makers of Spotfire, acquired Mathsoft. S-PLUS’s last release was in 2010; it was displaced by its open-source clone: R.

3. The Challenger Disaster

Perhaps the most famous catastrophe in which statistics played a part was the 1986 disintegration of the Challenger space shuttle over the Atlantic, which killed all 7 crew members. The cause of the break-up was the failure of an O-ring joint, which allowed hot pressurized gases to escape and destroy the structural connections among the shuttle’s rocket boosters and fuel tank. Owing to the low temperature at launch (39 degrees Fahrenheit), the O-ring had lost the elasticity it needed to adapt to launch stress and maintain a seal.

The possible failure of the O-ring at low temperature had been discussed and debated before the launch. Indeed, there had been non-catastrophic O-ring failures previously. The following scatterplot of launch temperature (x-axis) versus O-ring failures per flight (y-axis) was reviewed by a team of engineers:

The team concluded that there was no relationship between launch temperature and the number of O-ring incidents on a flight. An investigation after the disaster characterized the team’s view as

“Temperature data not conclusive on primary O-ring blow-by.”

This analysis looked only at the flights where incidents did occur, it left out the flights where failures did not occur. When you add in the latter, a different picture emerges. All the flights with no incidents occurred at temperatures above 65 degrees; all the flights below that temperature did have incidents. Clearly there is a relationship between temperature and O-ring failure. The Challenger flight was launched at a temperature of 39 degrees – the coldest ever.

Moral of the story:

Look at all the relevant data – don’t pick and choose.

Subscribe to the Blog

You have Successfully Subscribed!

By submitting your information, you agree to receive email communications from statistics.com. All information submitted is subject to our privacy policy. You may opt out of receiving communications at any time.

Categories

Recent Posts

  • March 9: Statistics and Data Science in Practice March 7, 2021
  • Feb 23: Statistics and Data Science in Practice March 5, 2021
  • Word of the Week – Ruin Theory March 4, 2021

About Statistics.com

Statistics.com offers academic and professional education in statistics, analytics, and data science at beginner, intermediate, and advanced levels of instruction. Statistics.com is a part of Elder Research, a data science consultancy with 25 years of experience in data analytics.

Latest Blogs

  • March 9: Statistics and Data Science in Practice
    March 7, 2021/
    0 Comments
  • Feb 23: Statistics and Data Science in Practice
    March 5, 2021/
    0 Comments
  • Word of the Week – Ruin Theory
    March 4, 2021/
    0 Comments

Social Networks

Linkedin-in
Twitter
Facebook-f
Youtube

Contact

The Institute for Statistics Education
4075 Wilson Blvd, 8th Floor
Arlington, VA 22203
(571) 281-8817

ourcourses@statistics.com

© Copyright 2021 - Statistics.com, LLC | All Rights Reserved | Privacy Policy | Terms of Use

By continuing to use this website, you consent to the use of cookies in accordance with our Cookie Policy.

Accept