The personal website of Matt Henderson.
14 October 2010
In his excellent book, “Innumeracy,” John Allen Paulos presents a number of surprising and important consequences of our general lack of understanding of the principles of mathematics; particularly, statistics, probability and the influence of large and small numbers.
In math, our intuition can’t always be trusted.
In a striking example, he considers how one should react to testing positive for cancer.
Assume that there is a test for cancer which is 98 percent accurate; i.e., if someone has cancer, the test will be positive 98 percent of the time, and if one doesn't have it, the test will be negative 98 percent of the time. Assume further that 0.5 percent one out of two hundred people actually have cancer. Now imagine that you've taken the test and that your doctor somberly informs you that you've tested positive. The question is: How depressed should you be? The surprising answer is that you should be cautiously optimistic.
So the question is, assuming you’ve tested positive for cancer (under these assumptions), how likely is it that you actually have cancer? As he demonstrates, it’s suprisingly low; the conditional probability that one has cancer, given that one tests positive, is only 20%.
The influence of sampling.
In another example, Paulos discusses filtering and the effect of sample-size on probability distributions.
When measuring a random variable, it turns out that in most situations, you don’t need to take very many samples to reasonably determine the average; often 30 samples will be enough. What changes with additional samples is the distribution the extreme values will be far more extreme.
An interesting manifestation of this, and the fact that most news broadcast filter the news on that which is “bad,” is that a 30 minute broadcast covering international news will seem worse, than a broadcast covering national news, which will seem worse than local broadcast, which will seem worse than a neighborhood broadcast.
Daily exposure to global news can, therefore, have a profound affect on our perception of the world. If we were only exposed to our local news, most of us would likely perceive the world to be a far more tranquil place.
I read this book by Paulos about three years ago, and subsequently stopped watching or reading the news cold turkey. Looking back, I believe I’ve benefitted from the absence of daily reports of murder, disaster and suffering. Furthermore, the sense of shock I now feel when I occasionally look at an international newspaper, serves as a cold reminder at how anesthetized my senses had become when exposed on a daily basis, and how, over time, I’d become unaware of that even happening.
Extension to social media.
Recently, I’ve been thinking about how Paulos’s lessons might be applicable to other areas in which we’re frequently exposed to filtered information on a global scope the obvious example being consumption of social media, like Twitter.
As I reflect on my personal consumption patterns, it’s clear that the slice of Twitterers that I follow on a daily basis mostly come from the contexts of design and online products areas in which I’m personally and professionally interested. I also observe that, for the most part, this slice represents the fat section of the long-tail of socially active participants in these industries and I guess that’s no surprise; the constituents of this section are who most of us follow.
Acknowledging the influence that daily exposure to this segment of the industry is likely to have, over time, on our perception of what’s important in these fields, and even what defines these fields, we should probably more often reflect on questions like the following:
In the same way that news tends to filter on the bad, on what dimensions does filtering occur in the context of social selection? Certainly we find great achievers among the fat-section of the long tail, but we also find first-movers in the social space, as well as those whose popularity apparently derives from being controversial.
How does achievement of influence today, in the context of social media, compare with the achievement of influence in the past? Are we better or worse off?
Analogous to the way that global sampling in the news space exposes us to greater extremes, which can distort our perception of reality, in what ways can the social sampling (geographically, culturally, etc.) brought to us by social media patterns distort our perceptions related to a particular field?
Can we trust our intuitions in these questions?
It’s some interesting food for thought.
(And in a note of irony, I notice that my blog footer suggests…)
Enjoy this article? — You can find similar content via the category and tag links below.
Questions or comments? — Feel free to email me using the contact form below, or reach out on Twitter.