I just read Super Crunchers, by Ian Ayres. The author, a law and business professor at Yale, presents dozens of examples of how computerized data crunching is changing business, politics, even teaching. He shows how computerized formulas routinely trump experts in predicting everything from what years will have good vintages to what unemployment policies are the most successful at helping people find jobs quickly.
The argument was compelling: in case after case, he showed how traditional experts were outdone by numbers. Baseball scouts, for instance, attend one game and believe that they can see something special that will identify a stand-out player. Yet Ayres points out that the difference between a .275 hitter and a .300 hitter is one hit every two weeks. This is a traditional stat any baseball fan follows, but then Ayres explains how a number cruncher named Bill James developed a regression formula to find how any given player actually contributed to runs created. He can now show not only how a players' individual stats add up, but how he contributes to the team's success.
Scouts, naturally, are less than enthusiastic about the new technique. Ayres goes on to point out how experts in their respective fields resist data crunching as it infringes on their terrain. If Ayres is right, the role of human expertise is quickly becoming obsolete. Intuition may still be needed, but in a vastly new way: he sees its role as finding explanations behind the numbers, and determining what factors to weight and how to weight them. Human expertise will be in developing new formulas to do the expert thinking for us.
I still have questions. I believe that skepticism, unlike cynicism, is a healthy thing. The book presented so many relevant data and seemed to address most of the immediate concerns or doubts that came to mind. I want to know more; I want to know HOW to do it myself, and not just why data crunching is changing the world. It inspired me into looking into some stats courses because, if Ayres is right, knowing how to crunch numbers grants a certain kind of power--the kind of power I crave--the power of knowing something of significance.
1 comment:
It doesn't make human expertise obsolete... you just have to have expertise in the right areas, like how to write those programs! :) I was in grad school before I really realized how all engineering is stats as well... that's how we figure out where satellites are. We observe it enough times to just get the best "guess" as to where it's going next, all based on stats.
So, you should take a stats class! :)
Post a Comment