This piece by Sharon Begley makes me rethink my desire for Newsweek to go bankrupt. It’s about a study of done and how to tell which ones are likely to be right:
The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit’s accuracy. And therein lay Tetlock’s first clue. The media’s preferred pundits are forceful, confident and decisive, not tentative and balanced. They are, in short, hedgehogs, not foxes.
[….]If there are three possibilities (say, that China will experience more, less or the same amount of civil unrest), throwing darts at targets representing each one produces a forecast more accurate than most pundits’. Simply extrapolating from recent data on, say, economic output does even better. But booking statistical models on talk shows probably wouldn’t help their ratings.
I went to a panel discussion by some poly sci professors the Friday before the election where they carried on about the Bradley effect and the vagaries of polls and how it wasn’t clear at all what would happen on Tuesday. My friend who was with me got worried and asked “is it really so up in the air?” I said “No, the election will go more or less exactly how Nate Silver is predicting it will go.” And it did.
Full disclosure: I found this article via Howie Kurtz.
FourtyTwo
But, but, but, the media is liberal!
Krista
Well, yes. Smart as hell and cute as a button? Ain’t nothin’ wrong with that.
I wonder, though — if Nate had his own show, would people listen? Or are we so dumbed down as a society that we’d tune out anybody who doesn’t speak in loud, declarative sound bites? He’d wind up being a modern-day Cassandra.
Evinfuilt
We the people are a lot smarter than those in the media give us credit. We actually have the attention span to listen to Obama answer a question, even though they don’t.
Mind you, I’m sure a very large demographic would prefer watching sports, so we need punditry to mimic sports, well that’s the media’s excuse for dumbing down their coverage.
Common Sense
@Evinfuilt:
Is anyone really surprised that TV pundits don’t know a damn thing? I’ll go back to the sports journalist comparison.
Note how often BP’s brilliant analysis got Silver a gig on ESPN. The success of PTI on ESPN shows exactly where TV execs’ priorities lie. They don’t want statistical analysis. They don’t see their role as one of spreading accurate or previously unknown information. They want to create controversy.
jenniebee
I think Nate has probably found his best medium already, one that’s tailored to his message. If television tends to favor soundbites and strong declarations and blogging tends to favor longer, more thoughtful discussions, then accept that no matter whom you put on television, television is going to be soundbiting them down. Wasting Nate Silver on TV is akin to making Andy Warhol work strictly in charcoal pencil.
Comrade Mary, Would-Be Minion Of Bad Horse
I’ve been right-braining with Flash all morning, so I’m failing to parse this: "It’s about a study of done and how to tell which ones are likely to be right:"
This is probably a pretty obvious typo, but take pity on me, eh?
BDeevDad
Ditto, did you see him on Colbert?
4tehlulz
@Krista: If he became an asshole like Rush or Keith, people would listen; people are entertained by assholes.
SpotWeld
Wait.. hedgehogs are forceful?
Zifnab
So they did a statistical analysis of Jim Crammer a while back. He is, apparently, worse than a coin flip. Like, if there were a Crammer fund, composed of only his stock advise, and you were to short it over the last few years, you’d make a killing.
At the end of the day, I think people do actually value factually accurate information so long as its not totally indigestible. People will do without your dog and pony show if you can actually give reliable information. And credibility is alluring. Dr. Phil wouldn’t be popular if everyone thought he was a quack. Martha Stewart wouldn’t have such a big following if her tips were totally impractical or her techniques never worked.
There is a demand for accuracy and reliability. It’s the supply side that’s lacking.
Zifnab
@BDeevDad: Oh shit, when was he on Colbert?
BDeevDad
@Zifnab: Before the election. Here you go.
c u n d gulag
Every time I see Nate or Krugman on I do a double-take.
How is it that rational, thoughtful people ever got on TV?
Usually, I see some spit-spewing moron demonstrating just how stupid stupid has become in the last 30 years.
And that’s pretty stupid…
jibeaux
Sharon Begley is a good science reporter. I get NW free (local public radio donation), and I usually find her pieces to be interesting, well-written, and good at the laymen’s explanation (which I need).
jibeaux
Oh, and seconded on Nate being cute as a button. He looks like he put on a suit and then shrunk a size or two, like in the movies when they age backwards but just a little bit, and there’s just something very adorable about that.
trizzlor
Maybe it’s too much to ask, but I’d like to see a searchable database of these pundit predictions and their accuracy – I think it’d be pretty fun to look up Kristol’s trail of death every once in a while.
Also, if individual pundits are wrong more times than random, their predictions could certainly be pooled to make a good overall prediction – even if it means taking the opposite of their advice.
Krista
And that’s why I keep coming back here. ;)
I’d say there’s a certain amount of thrust to a hedgehog…
Comrade Mary, Would-Be Minion Of Bad Horse
I swear to God, Krista, if you link a video to back up your last statement, there will be tears.
TenguPhule
Pleae keep TZ’s sex life out of this, Krista. ;P
nylund
Maybe they could keep all the present talking heads, but underneath their name on the screen could be some sort of accuracy rating. So like if Bill Kristol is talking, underneath him would be a (4%) so that you can still listen to him should you choose, but you’d also know he’s been wrong about 96% of his predictions. You could weight various predictions in a matter of importance. I know such a thing is awfully subjective, but things like, "there is no history of sectarian violence in Iraq," or whatever Kristol said, should affect his rating more than, say, predicting a running mate.
I’d suspect this would have a strong enough effect that the TV shows would start self-correcting by hiring people with better prediction rates.
Jon H
"Time to clone Nate Silver"
Replace Wolf Blitzer with eight Nate Silvers in an OCTOBOX.
Chris Andersen
I remember reading a comment by someone who was brought on some cable show to be an expert commentator on some issue. Right before the show went live, some breaking news hit. Since this pre-booked expert was the only person they had in studio, they put him on the air to discuss the breaking news.
Here’s the thing: the pre-booked expert knew NOTHING about the issues involved in the breaking story. The producer of the show knew this but put him on anyway. The pre-booked expert said, in response to every query from the host, that he didn’t know anything about it and therefore didn’t feel qualified to answer.
Result: the show was extremely awkward and the expert was never asked back.
Conclusion: TV only cares about guests who are compelling. Expertise is for seminars. The only experts TV cares about are those who are experts in sounding like they know what they are talking about in an interesting and compelling manner.
Chris Andersen
PS., If I could give one bit of advice to Krugman it would be to close his mouth when he isn’t talking. It makes him look dazed and confused when he isn’t responding to a question. Of course, maybe he is dazed and confused by the levels of stupidity around him.
The Cat Who Would Be Tunch
I’ve come to the conclusion that I’m in the wrong line of work. The only recession-proof job is being a professional pundit. I mean, what other job is there that promises you name recognition and money, irrespective of how many times or how badly you screw up? I’d love to be in a position where I can have the following conversation with my boss:
Me: Hey! Only 70% of my predictions were wrong this week!
Boss: Wow! Only 70%? That’s a new record.
Me: Yeah, although it’ll probably be a little lower next week.
Boss: No problem, here’s a raise for your hard work.
D.N. Nation
Ahh, the salad days of late October 2008, where every day you could go on to Instapundit, Gateway Pundit, Michael Barone’s blog, the Korner, etc., and see how THE POLLS HAD CLOSED, how THE RACE WAS STILL FLUID, how EVERYONE IS STILL UNDECIDED, how THE BRADLEY EFFECT WAS HAPPENING, how OBAMA IS LOSING IT, how COMEBACK MAC WAS BACK, how EVERYONE LOVES SARAH, how VOTERS ARE TURNING ON THE MEDIA, and how Wrightgate, Ayersgate, Cheesesteakgate, Bittergate, Murthagate, Plumbergate, Lipstickgate, Spreadthewealthgate, Coalgate, Bidengaffegate, Marxistgate, and on and on and on and on and on were going to be THE THING THAT SINKS OBAMA. This stuff, every hour, every day. The race was literally in the constant state of tightening.
Except when it wasn’t. Which was always. Silver looked at the data, showed exactly what McCain needed to do to ACTUALLY come back, and when it was clear this wasn’t going to happen, was frank about the man’s chances of victory…which was nil. As a result, you never saw a single wingnut pundit/hack/stenographer/thinker ever address Silver’s process outside of the occasional homina-homina-discounting-teh-Bradley-Thingy. Hell, I don’t think Glenn Reynolds linked to him even once…clearly Nate Silver isn’t the important political thinker Bob "Confederate Yankee" Owens is.
The Other Steve
This race is still close… There is still a chance John McCain could pull out an upset.
BDeevDad
This was also the reason he was not on the news where it had to stay close for ratings.
D.N. Nation
@The Other Steve: Exactly. That, all the time. The race was CONSTANTLY getting closer. Except, not in the slightest.
Medicine Man
I just imagine McCain trying to get a handle on the economy bailouts happening. Bumbling around… tilting an ear to the sage advise from his GOP comrades. Ugh. I think the US dodged a bullet.
DougJ
But if you had an arbitrarily large number of Andy Warhol clones, you might want one to work strictly in charcoal pencil.
Use your imagination, people!
Xanthippas
Well, let’s not forget about incentives. The only reason pundits thrive is because people tune into hear brash, confident, wrong-headed opinions that match their own. Most Americans don’t realize that if you intelligent discourse, blogs are the way to go.
Xecky Gilchrist
Nate Silver is a brilliant statistician – fivethirtyeight was my favorite site all through the election. Absolutely fascinating.
The only thing I find a bit off-putting about him is those posts he does about "if I were a Republican, I’d do thus and so because of this and that", where he’s assuming that Republicans study the world around them, reflect on it, and make rational decisions.
The Raven
One of the signs of real knowledge is usually less, clearer, and more direct expression; many people regard voluble, even aggressive, confidence as a reliable sign of knowledge, which it is not.
Krawk!
jenniebee
@DougJ:
If my imagination was that good then I would be Andy Warhol and Andy Warhol would be some random geek commenting on a blog instead of being dead, which I would be at that point. Or something.
Jeff Berardi
Anyone ever read Nate back when he was just at Baseball Prospectus? I’ve been telling everybody how great he is for at least half a decade. When I found out about 538… man… it was like finding out Jesus also knew karate.
And yes, we need to clone him, because I’m deeply concerned that his new career is going to eat into his old one. While forecasting the outcome of US elections is important and all, I could give a crap about all that stuff if it interferes with me getting an accurate projection of Ryan Howard’s home run production in ’09. Millions of fantasy teams hang in the balance, Nate! Make with the PECOTA cards already!
EnderWiggin
@trizzlor: i have been thinking about putting together a site like that for years.
Maybe the BJ community would be a good source of people to fill in the data.
Blue Raven
I think I might speak for a lot of hedgehog lovers when I say that comparing Coulter et alia to hedgies is a gross calumny on a darling animal whose points actually exist for a logical reason.