New Pollsters

Poll analysis sites put new spin on statistics
Reyhan Harmanci, SF Chronicle Staff Writer

Saturday, October 18, 2008

No matter which presidential candidate is elected on Nov. 4, there's already a clear winner in the 2008 campaign season - Nate Silver, 30, the founder of polling analysis Web site FiveThirtyEight.com.

In just a few months, Silver has gone from being an anonymous blogger to the toast of political junkies in red and blue states alike. The baseball statistician-turned-polling wizard has been heralded as a "rookie of the year" by Time magazine's Joe Klein, invited to spend election night with Dan Rather, profiled in New York and Newsweek magazines, and asked to appear on MSNBC with Keith Olbermann and "The Colbert Report" with Stephen Colbert. His 8-month-old site, run by a voluntary staff of three, is getting more than 600,000 hits a day. His Facebook fan group - the surest sign of Internet fame - features comments from readers in Australia and England.

Silver's FiveThirtyEight.com (named for the number of electoral votes) is one of a new breed of aggregated polling sites that collect, analyze and graph state and national polls. With the 24-hour news cycle grinding out an unprecedented amount of polling information, sites like FiveThirtyEight.com, Pollster.com, the Princeton Election Consortium (election.princeton.edu) and RealClearPolitics.com are giving voters new tools to interpret the tsunami of electoral data.

The poll aggregators cut through the maze of numbers with clear graphs, copious explanations of methodology and lively blog posts. And with the election only weeks away, even people with no knowledge of statistics are turning to the sites for guidance about which polls to trust.

The site purveyors are a mix of former pollsters, academics and statisticians. Each uses different mathematical models to contextualize the polling information. Some, like FiveThirtyEight.com, rate the pollsters on past performance and methodology; others find computing averages of numerous polls to be the most effective prediction technique. In the case of FiveThirtyEight.com, Silver's innovation was to add demographic data to polling numbers, which resulted in the accurate prediction that Sen. Barack Obama would capture Iowa in the Democratic primaries, something many pundits didn't think was possible.

Part of the reason aggregated polling sites have sprung up this election season is that, as San Francisco political consultant David Binder put it, "Every cycle, there are more and more polls. When I first started (20 years ago), there were only a couple of pollsters in the country." Pollster growth is "on a straight upward incline," added CBS polling head Kathy Frankovic.

Sea of information

This onslaught of information is keeping polling aggregators busy. Pollster.com founder Mark Blumenthal said he looks at about 16 new polls every day for the presidential election and six to seven for House and Senate races. Silver has posted ratings for 32 pollsters on his site and said that with the election only a few weeks away, he looks at 20 or 25 new polls on weekdays and five or 10 on weekends.

"I'm not sleeping much," Silver admitted.

Although the number of polls is growing, individual predictions by pollsters are not growing more accurate. In the Democratic primaries, for example, reputable surveyors like Gallup and Zogby failed to anticipate Sen. Hillary Rodham Clinton's win in New Hampshire. Structural issues, such as the growing number of cell phone-only households, are thought to hamper traditional polling techniques. And although Internet polling has become more popular, it is frowned upon by many pollsters who find the group too self-selected to provide an adequate random or representative sampling.

"One of the reasons I started FiveThirtyEight was because I thought there's more that could be done," said Silver, who was well known in sporting circles for his work at Baseball Prospectus, a statistical think tank that had success forecasting Major League Baseball winners and losers. "RealClearPolitics, for example, just takes an average of the polls. It has a strong brand, but if this is all they are doing - and they are getting that much traffic out of that - I thought we could do something more interesting and sophisticated."

Blogger's path

With no previous political experience, Silver began blogging under the name "Poblano" at the Daily Kos Web site this past winter. He launched FiveThirtyEight.com in March with a number of new tools that he invented to track and predict the presidential election in much the same way he had tracked and predicted baseball.

While Silver focuses on refining the predictive capabilities of the available polling data through complex mathematical machinations, other analysts have different goals. Sam Wang, a Princeton neuroscientist and founder of the Princeton Election Consortium, said, "What's needed is the simplest statistical measure that captures all of the data.

"We definitely have differing points of view about what's important to do with polling data," Wang said of his fellow aggregators. "My view is that, historically, state polls have been an extremely accurate predictor of the national race. By simply averaging them, you are right 98 percent of the time."

Wang also is skeptical of using polls as more than snapshots of a moment in time. For him, polling is better used to illuminate the present situation rather than to predict the future.

"This meta-analysis allows me to track the ups and downs. You can see, for instance, if it worked for McCain to add Palin to the ticket. You can see the results day by day, week by week. That's the power of it," Wang said.

Veteran pollster-turned-analyst Blumenthal, who began posting as "Mystery Pollster" at a Web site of the same name in mid-2004 before teaming up in 2006 with University of Wisconsin political science Professor Charles Franklin to create Pollster.com, takes a different approach. Pollster.com uses every available poll that claims to be a representative sample to create interactive graphs.

"We draw pictures," Blumenthal said. "We make it graphical. The way we do the chart, we plot every result as a dot for McCain or Obama, and then draw a regression trend line."

By presenting a large cross-section of polling information, Blumenthal aims to let readers manipulate the data themselves.

"We deliver on the mission to empower and put tools in the hands of the ordinary amateur," he said. "The hundreds of thousands of people who visit our site are really interested in what the polls have to say, rather than us being an officious oracle."

Some traditional pollsters are less than convinced by the methods of polling aggregators. There's an issue here - not all polls are created equal. Some have more rigorous methodology than others. You lose something by simply offering aggregations," said Frankovic.

Pollster John Zogby said he doesn't know FiveThirtyEight "from a hole in the wall" and found Pollster.com to be "very inside baseball for geeks." In general though, he said that "polling aggregation is a good thing" because it enriches political discourse.

"The big question is: The surveys are fallible, but by how much?" said Blumenthal. "We become better educated consumers to the extent that we understand (a poll's) limitations.


Online resources
Polling aggregate sites mentioned in this article:

-- www.FiveThirtyEight.com

-- www.Pollster.com

-- election.princeton.edu

-- www.RealClear Politics.com.