The always-worth-reading Dr. Thomas Sowell (one of my personal heroes) weighs in on misleading statistics. Excerpt:
Mark Twain famously said that there were three kinds of lies — “lies, damned lies, and statistics.” Since this is an election year, we can expect to hear plenty of all three kinds.
Even if the statistics themselves are absolutely accurate, the words that describe what they are measuring can be grossly misleading.
Household income statistics are an obvious example. When we hear about how much more income the top 20 percent of households make, compared to the bottom 20 percent of households, one key fact is usually left out. There are millions more people in the top 20 percent of households than in the bottom 20 percent of households.
The number of households is the same but the number of people in those households is very different. In 2002, there were 40 million people in the bottom 20 percent of households and 69 million people in the top 20 percent.
A little over half of the households in the bottom 20 percent have nobody working. You don’t usually get a lot of income for doing nothing. In 2010, there were more people working full-time in the top 5 percent of households than in the bottom 20 percent.
Household income statistics can be very misleading in other ways. The number of people per household is different among different racial or ethnic groups, as well as from one income level to another, and it is different from one time period to another.
The number of people per American household has declined over the years. When you compare household incomes from a year when there were 6 people per household with a later year when there were 4 people per household, you are comparing apples and oranges.
Now, I make my living teaching high-tech companies to solve problems. Part of that work involves showing high-tech company employees not only how to analyze whether the solutions to a given problem worked, but how to prove that those solutions worked. That, True Believers, takes statistics, and some knowledge of statistical analysis. Now I am not and never will be a statistical maven on the scale of Dr. Sowell, but I do know a few rules that must be applied if your analysis is to be meaningful:
- Compare apples to apples.
- Examine trends. A snapshot of data is interesting, but trends are evidence of effects.
- If your results do not prove your hypothesis, change your hypothesis; don’t “test until passed.”
As you can see from the examples Dr. Sowell provided, pols and their legacy media supporters are good at ignoring the rules of valid statistics. In other words: They are good liars, in the sense of Mr. Clemens’ quote: “Lies, damn lies, and statistics.”
