Wisdom of the JSM Crowd - 2017
In 2016, we conducted a guessing contest in homage to Sir Francis Galton's famous "Ox Weight Contest" (in which county fair contestants in the aggregate, correctly estimated the weight of a large ox - more below).
We asked JSM attendees to guess the amount of money in a glass bowl, and were struck by the extremely long right tail of this skewed distribution. The maximum guess ($2996) was 22 times the actual sum ($138). Moreover, the distribution was regularly shaped and looks like a typical member of a long-tailed family of distributions, e.g. exponential; the high guesses are not outliers.
2016 actual: $137
2016 mean guess: $253 (median guess $178)
2017 is the 100th anniversary of the Galton ox contest, and we reprised our money guessing contest, with a variation. Last year we had mostly one dollar bills, but two $5 bills and one $10 bill were mixed in, and at least one was always visible. This year we had only $1 bills, and we so informed contestants.
The results were considerably less skewed:
2017 actual: $117
2017 mean guess: $135 (median guess $123)
A second twist we tried was to allow contestants a second guess, after looking at three randomly selected prior guesses. 28 attendees tried a second guess, and those results were even more accurate:
2017 actual: $117
2017 mean second guess: $121
Sir Galton and the Ox
In case you've forgotten, Francis Galton observed a contest at a livestock show in 1907, in which attendees were invited to guess the dressed weight of an ox. Both the mean and the median of the guesses were spot on - within 1%. In our 2016 contest, the mean was off by 83%, and the median by 29%.
Speaking of the ox contest, a closer read of Galton's 1907 data shows that distribution of ox estimates was skewed as well, though not nearly as strongly, and his skew was to the left (the under-estimation errors were greater than the over-estimation errors). The magnitude of the errors was also much smaller than we experienced in our contest - the farmers' guess at the first quartile was short by only 4%.
Obviously the farmers were well familiar with oxen and experts at estimating their weights. The results of the contest even swayed Galton's political views in a more democratic direction, and in favor of widening the voting franchise.
Galton's story of the ox contest has been a staple of statistical lore over the past century, and served as an inspiration in James Surowiecki's popular book, The Wisdom of Crowds. But perhaps crowds are not so reliable, after all? Eric Steiner, writing in Wired in 2015, describes a money-bowl contest just like the ones we conducted. His results were similar to ours, and he titled his article "Turns Out the Internet is Bad at Guessing How Many Coins are in a Jar."
Want to be notified of future courses?Yes