Skip to content

Explore Courses | Elder Research | Contact | LMS Login

Statistics.com
  • Curriculum
    • Curriculum
    • About Us
    • Testimonials
    • Management Team
    • Faculty Search
    • Teach With Us
    • Credit & Credentialing
  • Courses
    • Explore Courses
    • Course Calendar
    • About Our Courses
    • Course Tour
    • Test Yourself!
  • Mastery Series
    • Mastery Series Program
    • Bayesian Statistics
    • Business Analytics
    • Healthcare Analytics
    • Marketing Analytics
    • Operations Research
    • Predictive Analytics
    • Python for Analytics
    • R Programming
    • Rasch & IRT
    • Spatial Statistics
    • Statistical Modeling
    • Survey Statistics
    • Text Mining and Analytics
  • Certificates
    • Certificate Program
    • Analytics for Data Science
    • Biostatistics
    • Programming for Data Science – R (Novice)
    • Programming for Data Science – R (Experienced)
    • Programming for Data Science – Python (Novice)
    • Programming for Data Science – Python (Experienced)
    • Social Science
  • Degrees
    • Degree Programs
    • Computational Data Analytics Certificate of Graduate Study from Rowan University
    • Health Data Management Certificate of Graduate Study from Rowan University
    • Data Science Analytics Master’s Degree from Thomas Edison State University (TESU)
    • Data Science Analytics Bachelor’s Degree – TESU
    • Mathematics with Predictive Modeling Emphasis BS from Bellevue University
  • Enterprise
    • Organizations
    • Higher Education
  • Resources
    • Blog
    • FAQs & Knowledge Base
    • Glossary
    • Site Map
    • Statistical Symbols
    • Weekly Brief Newsletter Signup
    • Word of the Week
Menu Close
  • Curriculum
    • Curriculum
    • About Us
    • Testimonials
    • Management Team
    • Faculty Search
    • Teach With Us
    • Credit & Credentialing
  • Courses
    • Explore Courses
    • Course Calendar
    • About Our Courses
    • Course Tour
    • Test Yourself!
  • Mastery Series
    • Mastery Series Program
    • Bayesian Statistics
    • Business Analytics
    • Healthcare Analytics
    • Marketing Analytics
    • Operations Research
    • Predictive Analytics
    • Python for Analytics
    • R Programming
    • Rasch & IRT
    • Spatial Statistics
    • Statistical Modeling
    • Survey Statistics
    • Text Mining and Analytics
  • Certificates
    • Certificate Program
    • Analytics for Data Science
    • Biostatistics
    • Programming for Data Science – R (Novice)
    • Programming for Data Science – R (Experienced)
    • Programming for Data Science – Python (Novice)
    • Programming for Data Science – Python (Experienced)
    • Social Science
  • Degrees
    • Degree Programs
    • Computational Data Analytics Certificate of Graduate Study from Rowan University
    • Health Data Management Certificate of Graduate Study from Rowan University
    • Data Science Analytics Master’s Degree from Thomas Edison State University (TESU)
    • Data Science Analytics Bachelor’s Degree – TESU
    • Mathematics with Predictive Modeling Emphasis BS from Bellevue University
  • Enterprise
    • Organizations
    • Higher Education
  • Resources
    • Blog
    • FAQs & Knowledge Base
    • Glossary
    • Site Map
    • Statistical Symbols
    • Weekly Brief Newsletter Signup
    • Word of the Week

Blog

Why AI Projects Fail: Type III Error

  • February 1, 2021
  • , 4:53 pm

We encountered “Type III error” when it turned out that most people answering our Puzzle question were, in fact, answering a different question from the one that was asked. Type III error is answering the wrong question, and it is a big factor in the failure of many AI projects.

We encountered “Type III error” when it turned out that most people answering our Puzzle question were, in fact, answering a different question from the one that was asked. Type III error is answering the wrong question, and it is a big factor in the failure of many AI projects.

Gartner Research estimates that 85% of AI and data science projects fail to be implemented.  Gerhard Pilcher, CEO of Elder Research (which owns Statistics.com), says that our implementation success rate is much higher – at over 65% – but he laments even the one-third of projects which are not put into production by / for the clients, saying those losses are preventable.

 Different variations of Type III error are often responsible for these failures.  Sometimes, a CEO will be seized by the idea that their company must commit itself to analytics and AI, and initiate projects to that end, without regard to a clear operational goal to be achieved. “They expect to dump data into a magical ‘black box,’ turn a crank, and generate insights,” says Pilcher. Other times, the business teams for which a project is intended (e.g., marketing or finance) want answers about revenue and profit, while the technical team will be evaluating the project by narrow metrics such as model accuracy.  Or the data science team is developing a model to answer a question that is relevant in the abstract, but not translatable to a form that the IT department can or will implement.

Detecting Fraud

One example of the benefits of learning to ask the right question comes in the area of fraud detection. To stay one step ahead of the cops, hackers and garden variety fraudsters are constantly changing their methods.  In predictive modeling, this can make it hard to describe fraud in terms of feature values, since those will change as criminal behavior changes. Non-fraudulent behavior, however, remains relatively consistent, particularly if you cast the feature net broader than simply transaction data.  So why not model non-fraudulent behavior, instead?

Josh Sullivan of Booz Allen, writing in the Harvard Business Review about a project with a large financial firm reported that they did just that, focusing on understanding and modeling compliant behavior.  As he put it, 

“We changed the nature of the question we were asking. Instead of, “How do we model bad?” we asked, “What if we modeled good?”

 By modeling good, one can focus attention on the “non-good,” and presume that it might be bad. Sullivan said that this approach enabled them to save their client over $1 billion by stopping massive fraud schemes.

For more about Type III error, including additional examples, see our blog on the subject from last year.

Subscribe to the Blog

You have Successfully Subscribed!

By submitting your information, you agree to receive email communications from statistics.com. All information submitted is subject to our privacy policy. You may opt out of receiving communications at any time.

Categories

Recent Posts

  • March 9: Statistics and Data Science in Practice March 7, 2021
  • Feb 23: Statistics and Data Science in Practice March 5, 2021
  • Word of the Week – Ruin Theory March 4, 2021

About Statistics.com

Statistics.com offers academic and professional education in statistics, analytics, and data science at beginner, intermediate, and advanced levels of instruction. Statistics.com is a part of Elder Research, a data science consultancy with 25 years of experience in data analytics.

Latest Blogs

  • March 9: Statistics and Data Science in Practice
    March 7, 2021/
    0 Comments
  • Feb 23: Statistics and Data Science in Practice
    March 5, 2021/
    0 Comments
  • Word of the Week – Ruin Theory
    March 4, 2021/
    0 Comments

Social Networks

Linkedin
Twitter
Facebook
Youtube

Contact

The Institute for Statistics Education
4075 Wilson Blvd, 8th Floor
Arlington, VA 22203
(571) 281-8817

ourcourses@statistics.com

© Copyright 2021 - Statistics.com, LLC | All Rights Reserved | Privacy Policy | Terms of Use

By continuing to use this website, you consent to the use of cookies in accordance with our Cookie Policy.

Accept