I was adjusting my list of featured articles tonight, and while reading my old articles I ran across a n example of a common error I once meant to discuss, but have since ignored. Like so many of the errors I have encountered, this is primarily an error of mistaking casue and effect, and like most such mistakes, it leads to some very mistaken conclusions.
The error in question occurs in my essay "To Correct Debra Saunders". In this essay she throws out a number of facts and figures, all intended to show how those who take on too much debt end up in dire financial straits.
On the face of it, this is unobjectionable, but the problem is that the raw numbers do not show which event preceded the other. While it is possible, even likely, that some may have taken on too much debt and thus fallen on hard times, is it also not likely that others, after finding themselves in dire straits started taking on more debt in an effort to get themselves out of the hole?
This problem is common in economic analysis, especially when carried out by politicians or journalists. Starting with a premise, such as "people who get in debt have problems", they find numbers that appear to confirm it, and look no deeper. They assume that debt leads to disaster and never consider what else the numbers might mean. They never ask if people don't perhaps end up in debt precisely because their finances are already bad.
And so we end up hearing that numbers "confirm" the position the writer already assumed to be true. Rather than an honest analysis of the figures, we get the numbers used as mere window dressing to support a point which was assumed before the first number was examined.
That is not how analysis should be done. Even if one approaches the figures with presumptions, one should always ask if there are other possibilities that explain them. Even a politician or journalist should not simply see numbers as a chance to add yet another proof to their argument.
My other essays on related themes are:
Great Moments in Leftist Journalism
A Most Dishonest Commercial
Correlation vs. Causation
Making Bad News of Good
How To Blame the Free Market
How to Create Poor People
Some of these relate directly to the problem of confusing cause and effect, others with ways we can misinterpret numbers or confuse ourselves regarding causation. All describe how those who start off looking for a specific result can convince themselves they have found it.
Originally posted in Random Notes on 2008/07/08.