Prof. David Unwin has guided, developed and taught the spatial analysis curriculum at Statistics.com since 2005. David lives in central England, about an hour north of the storied Rothamsted agricultural research center. Until his retirement in 2002, he was Professor of Geography at Birkbeck College, University of London, where he retains an Emeritus Chair inContinue reading “Instructor Spotlight: Prof. David Unwin”
Yearly Archives: 2019
Statistics in Agriculture: Encycloweedia
Weeds are big business – the global herbicide market is over $35 billion annually. Weeds are also big government (think “invasive species”). California’s listing of weeds is called Encycloweedia, and the state publishes a quarterly newsletter called Noxious Times. Colorado publishes a similar periodical, Invader. The weed-killer Roundup is the focus of lawsuits that illustrateContinue reading “Statistics in Agriculture: Encycloweedia”
Tensor
A tensor is the multidimensional extension of a matrix (i.e. scalar > vector > matrix > tensor).
Problem of the Week: Missing Data
Question: You have a supervised learning task with 30 predictors, in which 5% of the observations are missing. The missing data are randomly distributed across variables and records. If your strategy for coping with missing data is to drop records with missing data, what proportion of the records will be dropped? Is the assumption ofContinue reading “Problem of the Week: Missing Data”
Student Spotlight: Barry Eggleston
Barry Eggleston is a health research statistician who has worked on both clinical trials and observational studies, and is currently with RTI in North Carolina. In his early career, his work was solely designing and analyzing clinical trials using typical biostatistics methods ranging from t-test to survival analysis and mixed models. After moving to RTIContinue reading “Student Spotlight: Barry Eggleston”
A Deep Dive into Deep Learning
On Wednesday, March 27, the 2018 Turing Award in computing was given to Yoshua Bengio, Geoffrey Hinton and Yann LeCun for their work on deep learning. Deep learning by complex neural networks lies behind the applications that are finally bringing artificial intelligence out of the realm of science fiction into reality. Voice recognition allows youContinue reading “A Deep Dive into Deep Learning”
Industry Spotlight: Credit Scoring
In the U.S., credit scoring is dominated by three companies – Experian, TransUnion and Equifax, employing roughly 30,000 people. An important player in the scoring methodology is FICO, previously Fair Isaac Corporation, and the scores are typically called “FICO scores.” Credit scoring is the oldest application of predictive modeling, fulfilling a need that has beenContinue reading “Industry Spotlight: Credit Scoring”
Industry Spotlight: The IRS is Watching You
The IRS (U.S. Internal Revenue Service) has been using computers to choose tax returns for audit since 1962. Early on, the selection was rule-based, but the IRS turned to statistical modeling in 1969, using the oldest predictive analytics model in the toolbox – discriminant analysis. Discriminant analysis, a linear classification technique, was first proposed byContinue reading “Industry Spotlight: The IRS is Watching You”
Book Review: Weapons of Math Destruction
Cathy O’Neil’s Weapons of Math Destruction, when it was first published in 2016, sounded an early alarm about the big data algorithms and their potential for social evil. The cover is adorned with a robotic death’s head and the subtitle reads “How Big Data Increases Inequality and Threatens Democracy.” O’Neil’s book begins with stories thatContinue reading “Book Review: Weapons of Math Destruction”
Historical Spotlight: Alan Turing
80 years ago, in 1939, Alan Turing began work on the code-breaking system that would eventually prove key in helping Britain survive the German submarine threat in the Atlantic. Last month, the Turing Award in computer science prize (sometimes referred to as the “Nobel Prize of Computing”) was given to three researchers, Yann LeCunn, GeoffreyContinue reading “Historical Spotlight: Alan Turing”
Confusing Terms in Data Science – A Look at Synonyms
To a statistician, a sample is a collection of observations (cases). To a machine learner, it’s a single observation. Modern data science has its origin in several different fields, which leads to potentially confusing synonyms, like these:
Confusing Terms in Data Science – A Look at Homonyms and more Synonyms
To a statistician, a sample is a collection of observations (cases). To a machine learner, it’s a single observation. Modern data science has its origin in several different fields, which leads to potentially confusing homonyms like these:
Confusing Terms in Data Science – A Look at Synonyms, Homonyms and more
To a statistician, a sample is a collection of observations (cases). To a machine learner, it’s a single observation. Modern data science has its origin in several different fields, which leads to potentially confusing homonyms and synonyms, like these: Homonyms (words with multiple meanings): Bias: To a lay person, bias refers to an opinion about somethingContinue reading “Confusing Terms in Data Science – A Look at Synonyms, Homonyms and more”
Industry Spotlight: Package Delivery
Nothing better illustrates the encroachment of data science and analytics on the older “economy of tangible things” than the business of delivering packages. The use of analytics in package delivery is not new. Companies like UPS and Fedex are longtime users of operations research methods like optimization and simulation to route inter-city shipments, site newContinue reading “Industry Spotlight: Package Delivery”
Ethical Practice in Data Mining
Prior to the advent of internet-connected devices, the largest source of big data was public interaction on the internet. Social media users, as well as shoppers and searchers on the internet, make an implicit deal with the big companies that provide these services: users can take advantage of powerful search, shopping and social interaction toolsContinue reading “Ethical Practice in Data Mining”
Job Spotlight: Sports Statistician
The field of sports statistician is not exactly new; the American Statistical Association’s section on Sports Statistics was formed in 1992. Three of Statistics.com’s instructors have professional experience in sports statistics – Ben Baumer (SQL) served as statistician for the NY Mets, Stephanie Kovalchik (Meta Analysis in R) with Tennis Australia, and Joe Hilbe, whoContinue reading “Job Spotlight: Sports Statistician”
Industry Spotlight: Baseball – Opening Day & Statistics in Sports
The U.S. baseball season opens Thursday, March 28, and celebrates the 48th season of analytics in baseball, beginning with the founding of the Sabermetric Society in 1971 (the same year that Satchel Paige entered the Hall of Fame). Analytics has come a long way in sports, and now has its own conference, the MIT SportsContinue reading “Industry Spotlight: Baseball – Opening Day & Statistics in Sports”
Jaquard’s coefficient
When variables have binary (yes/no) values, a couple of issues come up when measuring distance or similarity between records. One of them is the “yacht owner” problem.
Darwin’s Legacy in Statistics
Charles Darwin, the most famous grandson of the Enlightenment thinker Erasmus Darwin, published his ground-breaking theory of evolution, “The Origin of Species,”160 years ago. Another grandson of Erasmus, Francis Galton, became one of the founding fathers of statistics (correlation, the “wisdom of the crowd,” regression and regression to the mean are all Galton’s ideas). HeavilyContinue reading “Darwin’s Legacy in Statistics”
Industry Spotlight: Customer Segmentation
Are you “young and rustic?” Or perhaps a “toolbelt traditionalist?” These are nicknames given to customer segments identified by market research firm Claritas, with its statistical clustering tool. Long before the advent of individualized product recommendations, business sought to segment customers into distinct groups on the basis of purchase behavior, demographic variables, and geography, toContinue reading “Industry Spotlight: Customer Segmentation”