Conditional Probability Word of the Week

QUESTION:  The rate of residential insurance fraud is 10% (one out of ten claims is fraudulent).  A consultant has proposed a machine learning system to review claims and classify them as fraud or no-fraud.  The system is 90% effective in detecting the fraudulent claims, but only 80% effective in correctly classifying the non-fraud claims (it mistakenly labels one in five as "fraud").  If the system classifies a claim as fraudulent, what is the probability that it really is fraudulent?

SOLUTION

This is a problem in conditional probability. (It?s also a Bayesian problem, but applying the formula in Bayes Rule only helps to obscure what?s going on.)  Consider 100 claims. 10 will be fraudulent, and the system will correctly label 9 of them as “fraud.” 90 claims will be OK, but the system will incorrectly classify 18 (20%) as “fraud.”  So a total of 27 claims have been labeled as fraudulent, but only 9 of them, 33%, are actually fraudulent. Translating into probability terms:

Given that a claim has been labeled as fraudulent, the probability is 0.33 that it is actually fraudulent.

Or, in symbols:

P(fraud|”fraud” label) = 0.33   (the vertical bar means “given that.”)