Nothing better illustrates the encroachment of data science and analytics on the older “economy of tangible things” than the business of delivering packages. The use of analytics in package delivery is not new. Companies like UPS and Fedex are longtime users of operations research methods like optimization and simulation to route inter-city shipments, site newContinue reading “Industry Spotlight: Package Delivery”
Monthly Archives: March 2019
Ethical Practice in Data Mining
Prior to the advent of internet-connected devices, the largest source of big data was public interaction on the internet. Social media users, as well as shoppers and searchers on the internet, make an implicit deal with the big companies that provide these services: users can take advantage of powerful search, shopping and social interaction toolsContinue reading “Ethical Practice in Data Mining”
Job Spotlight: Sports Statistician
The field of sports statistician is not exactly new; the American Statistical Association’s section on Sports Statistics was formed in 1992. Three of Statistics.com’s instructors have professional experience in sports statistics – Ben Baumer (SQL) served as statistician for the NY Mets, Stephanie Kovalchik (Meta Analysis in R) with Tennis Australia, and Joe Hilbe, whoContinue reading “Job Spotlight: Sports Statistician”
Industry Spotlight: Baseball – Opening Day & Statistics in Sports
The U.S. baseball season opens Thursday, March 28, and celebrates the 48th season of analytics in baseball, beginning with the founding of the Sabermetric Society in 1971 (the same year that Satchel Paige entered the Hall of Fame). Analytics has come a long way in sports, and now has its own conference, the MIT SportsContinue reading “Industry Spotlight: Baseball – Opening Day & Statistics in Sports”
Jaquard’s coefficient
When variables have binary (yes/no) values, a couple of issues come up when measuring distance or similarity between records. One of them is the “yacht owner” problem.
Darwin’s Legacy in Statistics
Charles Darwin, the most famous grandson of the Enlightenment thinker Erasmus Darwin, published his ground-breaking theory of evolution, “The Origin of Species,”160 years ago. Another grandson of Erasmus, Francis Galton, became one of the founding fathers of statistics (correlation, the “wisdom of the crowd,” regression and regression to the mean are all Galton’s ideas). HeavilyContinue reading “Darwin’s Legacy in Statistics”
Industry Spotlight: Customer Segmentation
Are you “young and rustic?” Or perhaps a “toolbelt traditionalist?” These are nicknames given to customer segments identified by market research firm Claritas, with its statistical clustering tool. Long before the advent of individualized product recommendations, business sought to segment customers into distinct groups on the basis of purchase behavior, demographic variables, and geography, toContinue reading “Industry Spotlight: Customer Segmentation”
Industry Spotlight: CROs
CRO’s, or contract research organizations, are a $40 billion industry, growing at close to 12% per year. They provide contract services to the pharmaceutical industry, including statistical design and analysis, laboratory services, administration of clinical trials, and monitoring of drugs once they are on the market. Developing a new drug and bringing it to marketContinue reading “Industry Spotlight: CROs”
Handling the Noise – Boost It or Ignore It?
In most statistical modeling or machine learning prediction tasks, there will be cases that can be easily predicted based on their predictor values (signal), as well as cases where predictions are unclear (noise). Two statistical learning methods, boosting and ProfWeight, use those difficult cases in exactly opposite ways – boosting up-weights them, and ProfWeight down-weightsContinue reading “Handling the Noise – Boost It or Ignore It?”
Problem of the Week: Probability
Your country is at war, and an enemy plane has crashed on your territory. It bears the number 60, and a spy has told you that the aircraft are numbered serially. Can you make a guess about the total number of aircraft the enemy has produced? Solution: This problem is one of those published byContinue reading “Problem of the Week: Probability”
Rectangular data
Rectangular data are the staple of statistical and machine learning models. Rectangular data are multivariate cross-sectional data (i.e. not time-series or repeated measure) in which each column is a variable (feature), and each row is a case or record.
“Defiant” Supervision
How did the phrase “defiantly recommend”, as in “I defiantly recommend this product,” come into common usage on the internet? The answer is a good look inside the workings of supervised learning. Supervision, generally from humans, is instrumental in much of statistical and machine learning. Google’s precise search algorithms are not public, but the generalContinue reading ““Defiant” Supervision”