Wednesday, October 30, 2013

What I Need From Statisticians - Nate Silver

  • The average is still the most useful statistical tool ever invented. One of Silver’s main frustrations with the coverage of the election and the polls last year. Journalists do not focus on the average person but prefer a better story about an outlier which tells a better narrative. The bias can be quite explicit. People in politics can be willing to cherry-pick the data and not be apologetic for it. He respects the “lowly average as it performs almost as well as more complex methods but also because it serves as a litmus test for whether the journalist is worth their statistical salt or not”.
  • Know thy priors. Silver uses the Bayesian approach towards statistics. Methods can be abused but that is true with any approach, including Bayesian but it offers a more coherent, philosophical and sophisticated way in which to look at the world and the Bayesian method would be useful for journalists to use as well, in particular prior beliefs and bias.
  • The word ‘complex’ isn’t always a complement. When a journalist explains that something is complex, he/she may be unintentionally letting the reader know that he/she does not understand. If a statistician did this, it would be even more concerning! In this, Silver sees a lot of parallels between the role of a statistician and that of a journalist. A journalist has to take a complex set of facts and convey some understanding of them to the broader public – which details are most essential and which can be left out?
  • Insiderism is the enemy of scientific objectivity. Silver was fond of some of his reception from critics that nerds are taking over the world but it is an over-simplification. Inside information is expensive and takes a long time to cultivate. Such information may not be very reliable once you receive it. There are very talented reporters who can see through the charade but there are others who ‘do not have a very good BS detector and tend to follow the herd.’
- More Here

No comments: