Text analytics or text mining is the natural extension of predictive analytics, and Statistics.com’s text analytics program starts Feb. 6. Text analytics is now ubiquitous and yields insight in: Marketing: Voice of the customer, social media analysis, churn analysis, market research, survey analysis Business: Competitive intelligence, document categorization, human resources (voice of the employee), recordsContinue reading “Course Spotlight: The Text Analytics Sequence”
Yearly Archives: 2014
Course Spotlight: Constrained Optimization
Say you operate a tank farm (to store and sell fuel). How much of each fuel grade should you buy? You have specified flow and storage capacities, constraints on what types of fuels can be stored in which tanks, prior contractual obligations about minimum monthly deliveries and incoming supplies, plus the opportunity to sell onContinue reading “Course Spotlight: Constrained Optimization”
College Credit Recommendation
Statistics.com Receives College Recommendation from the American Council on Education (ACE) College Credit Recommendation for Online Data Science Courses from The Institute for Statistics Education at Statistics.com LLC The American Council on Education‘s College Credit Recommendation Service (ACE CREDIT) has evaluated and recommended college credit for 5 more of The Institute for Statistics Education atContinue reading “College Credit Recommendation”
Week #48 – Structured vs. unstructured data
Structured data is data that is in a form that can be used to develop statistical or machine learning models (typically a matrix where rows are records and columns are variables or features).
Big Data and Clinical Trials in Medicine
There was an interesting article a couple of weeks ago in the New York Times magazine section on the role that Big Data can play in treating patients — discovering things that clinical trials are too slow, too expensive, and too blunt to find. The story was about a very particular set of lupus symptoms,Continue reading “Big Data and Clinical Trials in Medicine”
Word #39 – Censoring
Censoring in time-series data occurs when some event causes subjects to cease producing data for reasons beyond the control of the investigator, or for reasons external to the issue being studied.
Industry Spotlight: The brand premium for Chanel and Harvard
The classic illustration of the power of brand is perfume – expensive perfumes may cost just a few dollars to produce but can be sold for more than $500 due to the cachet afforded by the brand. David Malan’s Computer Science course at Harvard, CSCI E-50, provides an interesting parallel in the education world. It’sContinue reading “Industry Spotlight: The brand premium for Chanel and Harvard”
Twitter Sentiment vs. Survey Methods
Nobody expects Twitter feed sentiment analysis to give you unbiased results the way a well-designed survey will. A Pew Research study found that Twitter political opinion was, at times, much more liberal than that revealed by public opinion polls, while it was more conservative at other times. Two statisticians speaking at the Joint Statistical MeetingsContinue reading “Twitter Sentiment vs. Survey Methods”
Internet of Things
Boston, August 3 2014: Bill Ruh, GE Software Center, says that the Internet of Things, 30 billion machines talking to one another, will dwarf the impact of the consumer internet. Speaking at the Joint Statistical Meetings today, Ruh predicted that the marriage of the IoT and analytics will yield $1 trillion in savings or productivityContinue reading “Internet of Things”
Work #32 – Predictive modeling
Predictive modeling is the process of using a statistical or machine learning model to predict the value of a target variable (e.g. default or no-default) on the basis of a series of predictor variables (e.g. income, house value, outstanding debt, etc.).
Week #29 – Goodness-of-fit
Goodness-of-fit measures the difference between an observed frequency distribution and a theoretical probability distribution which
Week #23 – Adjacency Matrix
An adjacency matrix describes the relationships in a network. Nodes are listed in the top..
Convoys
Ever wonder why, in World War II, ships in convoys were safer than ships traveling on their own? Most people assume it was due to the protection afforded by military escort vessels, of which there was a limited supply (insufficient to protect ships traveling on their own). Actually, most of the benefit came from theContinue reading “Convoys”
Dialects
When talking to several people, do you address them as “you guys”? “Y’all”? Just “you”? And is the carbonated soft drink “soda” or “pop?” Maps based on survey responses to questions like this were published in the Harvard Dialect Survey in 2003. Josh Katz took the data and produced extended visualizations and, last month, aContinue reading “Dialects”
Needle in a Haystack
What’s the probability that the NSA examined the metadata for your phone number in 2013? According to John Inglis, Deputy Director at the NSA, it’s about 0.00001, or 1 in 100,000. A surprisingly small number, given what we’ve all been reading in the media about NSA’s massive data collection effort. Of course, that’s an unconditionalContinue reading “Needle in a Haystack”