Fundamental divergence; oddity or realignment

There have been two interesting news stories on elections in the past week as well as an interesting inside baseball geek out concerning how to model and predict Senate elections that could be either interesting outliers, or harbingers of change.

The two interesting stories are the Democratic Parties of Kansas and Alaska happily seeing their preferred candidates for Senate and Governor respectively drop out of the race. There were no mysterious revelations of hookers, blow, green balloons, or toe tapping in the restroom. There were no plane crashes, there were no children of the candiddates being diagnosed with cancer.

Instead, the candidates dropped out in Kansas and formed a fusion/unity ticket to allow independent candidates who are polling well to be the primary opposition to Republican incumbents in deep red states. The basic thrust is that Senator Roberts and Governor Parnell are reasonably unpopular with the general electorate but could very easily cobble together a coalition of 43% of the voters. 43% is usually more than enough to win a plurality in a three way race, while 43% is a big loss in a two way race. The bet is that the independent candidates have a much higher probability of putting together a plurality or even better a clear majority coalition against the incumbent.

The basis of the bet is that both independents are former Republicans who look at the deep-red strains of the Republican Party and think they are sufficiently bat-shit insane that it was worth running against Republican incumbents. In Kansas, this has been a long tradition where the electorate has been split into nearly even chunks of Teabaggers/extreme conservatives, moderate Republicans and then a wide array of Democrats of various flavors. Democrats could win state wide office with good candidates who could pick up a good chunk of the moderate Republicans who were momentarily disgusted at the Teabaggers. It is a long and successful strategy. Democratic success in Alaska in the past generation has either counted on a felony conviction (later overturned) or a split Republican Party for any state wide wins.

If Democrats can successfully engage in a strategy of being the party of the sane and continue to pick up former Republicans (such as John Cole) without losing significant elements of the current Democratic base, is that the start of a realignment?

The other big, and geeky debate that I’ve been paying attention to has been the poll aggregating and prediction site differentials.

The poll aggregators are showing significantly better results for Democrats than the sites whose models including significant ‘fundamental’ weights. Sam Wang at Princeton Election Consortium is a aggregator with no special sauce predictor and has a good post on this:

I categorized models as “Fundamentals-based (Type 1)” and “Polls-based (Type 2)”. The major media organizations (NYT, WaPo, 538) have all gone with a hybrid Type 1/Type 2 approach, i.e. they all use prior conditions like incumbency, candidate experience, funding, and the generic Congressional ballot to influence their win probabilities — and opinion polls. What does that look like…

Senate Democrats are doing surprisingly well. Across the board, Democratic candidates in the nine states above are doing better in the polls-only estimate than the mainstream media models would predict. This is particularly true for Alaska, Arkansas, and North Carolina….

It is nearly Labor Day. By now, we have tons of polling data. Even the stalest poll is a more direct measurement of opinion than an indirect fundamentals-based measure. I demonstrated this point in 2012, when I used polls only to forecast the Presidency and all close Senate races. That year I made no errors in Senate seats, including Montana (Jon Tester) and North Dakota (Heidi Heitkamp), which FiveThirtyEight got wrong.

The fundamental predictors put a significant thumb on the scale by looking at factors such as incumbency, cash raised, candidate quality and the generic tilt of a state. These models would say that a generic Republican in Arkansas (a state that went big time for Romney and it generally conservative) should be significantly advantaged over a generic Republican candidate in Maine. The pure poll aggregators basically say that all of the fundamentals will be baked into the polls by the summer before the election, so polls tell a simpler story with far fewer confounding variables than a fundamental based approach.

If my memory serves me right, the Democrats have signficantly overperformed the fundamentals at the Senate level in 2010 (Nevada, Colorado, Delaware), 2012 (Indiana, Montana, North Dakota, Missouri) and seemingly so in 2014 at this time. John Kerry also overperformed his fundamentals in 2004 although it was an insufficient overperformance for a win. If my memory is right, is there a small but consistent error factor in the fundamental measures that is failing to pick up on the basic fact that there is a slight but persistant Democratic candidate quality edge that is being driven by the basic fact that the Republican primary electorate is batshit insane? And that primary electorate has a persistant ability to advance candidates who are weaker than they “should” be because they turn off some marginal members of the typical Republican winning coalition into either non-voters or hold your nose Democratic voters?

Share On Facebook
Share On Twitter
Share On Google Plus
Share On Pinterest
Share On Reddit

32 replies
  1. 1
    wmd says:

    GoTV is going to be crucial to get the “hold your nose” voters to the polls and vote against the batshit crazy candidate.

  2. 2
    Bobby Thomson says:

    The problem is that one of the fundamentals is this being an off-year election, which means that Democratic turnout will suck ass. In 2006, the Mark Foley scandal also depressed Republican turnout, but in 2010, Republicans showed up. Had it not been for a few extreme candidates in 2010 in the examples you give, it would have been even worse. This year, Republicans are not nominating the Christine O’Donnells.

    I tend to agree with Wang that fundamentals should be incorporated into polls, but it’s still too early for polls to mean much. We don’t really know who likely voters are going to be yet, and polls of all registered voters are meaningless in an off-year election.

  3. 3
    NotMax says:

    The media gravitating towards a horde race scenario?

    Hoocoodanode ?

  4. 4
    NotMax says:

    Heh. Call that a WoW typo. Of course, should read ‘horse race.’

  5. 5
    Valdivia says:

    the Independent in Kansas is actually a former Dem though he donated to Scott Brown in 2012. I think he would be more of a typical Mod Dem than a sane Republican. But a hearty yes to bringing on board all the same people.

  6. 6
    John M. Burt says:

    In 1980, the Democratic party was the Republican party of 1960. It only makes sense for them to complete their transformation into the Republican party of 1980.

    I agree that the best hope for the nation is that the Republican party of the 2010s will complete its slide into irrelevance before the damage it is doing to the country will be fatal, but what is most urgently needed is for a new party to appear on the left.

    So yes, let’s turn Kansas Blue, but only to give us a little time in which to turn the coasts Green.

  7. 7
    NotMax says:

    @Valdvia

    He has already laid out his stance as being that he will most likely caucus with whoever has the majority (or, in the event of a 50-50 split, translates as *buzzword buzzword* with Republicans),

  8. 8
    Iowa Old Lady says:

    @NotMax: Hee. I was sitting here trying to make “horde race” work. I pictured a buncha candidates all running at once, probably swinging axes.

  9. 9
    gene108 says:

    @John M. Burt:

    The Democrats made Civil Rights a part of their Party’s platform, I believe, back in 1948. They have been way ahead of Republicans on so many issues, for such a long time, branding Dems as Republican-lite is silly.

  10. 10
    rikyrah says:

    HEY KAY!!!

    …………

    Judge orders expanded Ohio early voting schedule

    By Associated Press September 4 at 1:55 PM

    COLUMBUS, Ohio — A federal judge on Thursday temporarily blocked an Ohio law that scales back early voting, ordering the state’s elections chief to set additional voting times just ahead of the fall election.

    The ruling from U.S. District Judge Peter Economus comes in a lawsuit filed by civil rights groups, several predominantly black churches and others challenging two early voting measures in the perennial presidential battleground state.

    One is a directive from Republican Secretary of State Jon Husted that established uniform early voting times and restricted weekend and evening hours. Another is a GOP-backed law that eliminates golden week, when people could both register to vote and cast ballots. Without those days, early voting would typically start 28 or 29 days before Election Day instead of the prior 35-day window.

    Ohioans can vote absentee by mail or in person.

    http://www.washingtonpost.com/.....story.html

  11. 11
    MattF says:

    I think part of the problem is that Republicans always have a carefully designed and comprehensive narrative about how they are going to win. Deliberately or not, pollsters are affected by this. It’s like the problems with testing for ESP– the subtle fallacy is that you stop testing when the data support your preconceptions. It’s very hard to eliminate this kind of bias, and in the polling world, where it’s all about the horse race, you’re not really motivated to try.

  12. 12
    Iowa Old Lady says:

    Early voting in Iowa actually starts three weeks from today. Bruce Braley needs to get his butt in gear.

  13. 13

    Time to Freep a poll! FWIW!

    It’s asking do you support the striking fast food workers. So far the Yesses are losing.

  14. 14
    rikyrah says:

    these are some evil ass children raised by evil ass parents

    ………………..

    Bullies Dump Feces, Urine on Autistic Boy in Evil Ice Bucket “Prank”

    Police are investigating after a group of teens in Bay Village, Ohio, allegedly tricked an autistic 15-year-old into creating an ice bucket challenge video, then dousing him with urine, feces, spit, and cigarette butts.

    The video was filmed on the boy’s cell phone and posted to Instagram, according to his mother, Diane. (Local news outlet WJW withheld the family’s last name to protect the boy’s identity.)

    The victim’s brother, Jacob, told WJW:

    I mean, the first thing that popped into my mind was like, why could someone – how could someone do this? How could someone really be this cruel to someone?

    In the video, the boy can be seen standing in a driveway, wearing only his underwear, and the liquid that comes splashing down on him appears to come from the roof of a garage. According to WJW, the family asked that the clip be broadcast, “to make other parents aware of bullying.”

    http://gawker.com/bullies-dump.....29/+cushac

  15. 15
    Richard Mayhew says:

    @Bobby Thomson: But the composition of the off-year electorate should (especially by now) be baked into the polls of likely voters and is partially baked into polls of registered voters. The question I have is the diveregence between fundamentals and results — is it a one-off or have the fundamentals gently changed over the past three or four election cycles?

  16. 16
    Enhanced Voting Techniques says:

    It might be worth noting that in 2010 the conservatives were all stoked by the rage that the Left was right all along and that’s faded. These elections that mentioned as toss up/ GOP loss the Republican is an incumbent and a banner bearer of the wakyloon Right. Obamacare is almost a non issue now, the Right had their governments shut down and didn’t like it, their attempt to turn immigration into an issue flopped, it’s hard to believe that ISIS is really going to have much legs, considering the GOP can’t even articulate anything beyond “Obummer is wrong!” And who knows, the GOP is plenty stupid enough to try yet another shut down next week.

    Personally, I wouldn’t be to surprised to see a lower than normal voter turn out of the GOP this November.

  17. 17
    jl says:

    I think Wang discussed two distinct issues that he does not really separate clearly in his post. The first is whether it is best to rely on polls alone or include a predictive model that includes information on non-polling information (such as recent economic performance, incumbency) hat has shown predictive power in modelling past elections. The other is whether it is best to build models that assume each individual election is an independent experiment unrelated to other elections, or include information about common factors that will produce correlations between individual elections.

    Regarding the first issue, think that there are two opposing factors. If you use polling information alone, you have to deal with the ‘talk is cheap’ issue, and the fact that the opinions people give in surveys often is poor guide to what they will actually do. They may have unconsciously made up their mind in their gut, but do not consciously realize it yet.

    The solution to that is to include information about the past predictive performance of more objective measures of what people have actually done in the past (which Wang, and I guess others, call ‘fundamentals’). I think that gives these more objectively measurable factors that can be used in a model too much dignity. I don’t think anyone really can reliably identify a set of factors they can justify as being real ‘fundamentals’ of human decision making. Really, they are just whatever collection of statistics that happen to be lying around with cheaply available observations .over a large number of election cycles that have shown good predictive performance in the past. Some election gurus like Sabato and a macroeconomist whose name escapes me at the moment claim to have identified the real fundamentals, but they fiddle so much with their models, and so much of what they do is a black box, it is really unknown whether they have a stable model of fundamentals or not.

    But I think the big problem with the fundamental models is that the predictive error intervals for future individual observations is so wide that with these models that they are pretty unreliable. The usual trick is to use the predictive intervals for the expected outcome, which is narrower, and fine for, say, a fatcat health insurance oligarch like Mayhew who can amortize gains and losses over thousands of independent outcomes. But if you are going to do real decision analysis, the costs and benefits over the next two years depend on the expected outcome of this individual set of elections, not some hypothetical set of hundreds of 2014 elections. So I think a the predictive interval or individual observations is more relevant, and that means that the including the fundamentals is not as valuable as some people think. I think Nate Silver has some posts on his old 538 site that explain this point well.

    I think it is reliably known that the cheap talk polls do tend to converge to the the fundamental models as we get closer to election day, so Sam Wang’s point that it is past Labor Day, so not a good idea to dismiss polls only approach is a good one.

    The issue of whether to model each election as an independent experiment or they should be modeled as have substantial correlations induced by common factors is very difficult, maybe impossible, to determine from standard statistical measures using only in-sample data used for estimation of the model. It is known that out-of-sample predictive performance of a stable model (with strong emphasis on STABLE MODEL that has not been repeatedly fiddled with between predictions) can give some information on this question. I think Sam Wang has a much better record on that point than others, so if he thinks common factors are important, I would go with him.

    I think Wang got a little sloppy in saying that assuming each election is an independent experiment can only introduce nose. You really cannot know that from any standard statistical procedure (at least I don’t know of any consensus on how to do that with standard in-sample statistical tests and measures). It might be true, it might not be, and out-of-sample predictive performance is the only way to get a good read on that issue. At an elementary level, this issues mainly comes up in panel data analysis in economics so a good textbook on that should give details, Maybe some math maven will come along who can give a good reference in another field.

  18. 18
    Kay says:

    @rikyrah:

    :) I just saw it and then I saw your comment!

    It’s even better than it looks, because of this, Section 2. As you know Roberts gutted Section 5, so 2 is the way in:

    More broadly, the courts are split over how to interpret the remaining provisions of the Voting Rights Act in the wake of the Supreme Court gutting a key part of the law last June. This is the first time a court has struck down limits on early voting under Section 2 of the VRA.
    A Bush-appointed judge recently denied a preliminary injunction to block North Carolina’s cuts to early voting and the elimination of same-day registration, a lawsuit similar to the one in Ohio. A Wisconsin judged blocked the state’s voter ID law under Section 2, while a similar trial is currently underway in Texas.

    So two for us under Section 2 and one for them, one not decided yet. The trial in Texas started Tuesday:

    In essence, then, Judge Gonzales Ramos, an Obama appointee, is being asked not just to rule on a high-stakes issue, but to answer a profound and controversial question: How much of today’s racial inequality is the result of past discrimination?

    It’s nice to have Obama-appointed judges in the mix, now, huh?

  19. 19
    FlipYrWhig says:

    @John M. Burt:

    what is most urgently needed is for a new party to appear on the left.

    I don’t see how a new party to the left of the Democrats does much. Even if you could somehow get their support up to 20-30%, and drive the support for the right-wing crazies down to 30-40% of the vote, which is a Pollyanna-Brand Best Case Scenario, you’ll still end up with a breakdown like 25-40-35 in favor of the Centrist Party. So then you might have a coalition between the Left and the Center–but that’s pretty much what we have now, only we call it one party rather than two. There just aren’t enough lefties for there to be a lefty majority party.

  20. 20
    cmorenc says:

    There are two potential interrelated “wild cards” in the 2014 elections that may not yet be baked into the polling:
    1) To what extent will Ferguson (and publicity over other recent incidents of brutal mistreatment of minorities) induce democratic-leaning minorities to turn out in significantly greater proportions in 2014 than has been typical in other off-year elections? And even if so, to what extent will this effect be successfully dampened by increased voting restrictions in states under current control by GOP legislatures, and to what extent will this suppression in turn be offset by increased “get out the vote” activism within minority communities to help constituents ovecome these restrictions?
    2) To what extent has the more generalized democratic-leaning electorate effectively absorbed the lesson of 2010, which is that if they stay home out of indifference or disaffection with perceived imperfections in the democratic nominees for office – they effectively increase the influence of hard-right voters and the likelihood that odiously right-wing candidates will win elections (and control of government)?

    A significant vulnerability of polling, even polling aggregates, is that the validity of polling depends on having an accurate turnout model. What is the effect if assumptions based on previous elections are now off even by five or ten percent on the democratic-leaning side? OTOH, turnout assumptions on the GOP-leaning side, based on 2010 or 2012, already more or less maximize the likely potential size of that portion of the electorate.

  21. 21
    mb says:

    Kind of disheartening that the big, exciting news is dems are dropping out of races. I understand all the reasons, but it doesn’t feel like a sign of a party in ascension. There are days when I despise the Democratic Party — today it’s more like disdainful pity. If the Rs weren’t knee-walking nuts we’d have no chance at all.

  22. 22
    jl says:

    @mb: I agree. There is something disheartening about it. But some wise man once said that politics is the art of the impossible. And Democrats, and the forces of sanity in general, need to concentrate more on the “‘just win, baby’ aspect of politics. The GOP sure does.

  23. 23
    Marc says:

    @John M. Burt:

    I agree that the best hope for the nation is that the Republican party of the 2010s will complete its slide into irrelevance before the damage it is doing to the country will be fatal, but what is most urgently needed is for a new party to appear on the left.

    Wow, only six comments in before the usual progressive suicide note turns up.

    Note that what Richard describes is the exact opposite of a rift opening up on the left, and it may be the thing that keeps control of the Senate in Democratic (read: sane) hands.

  24. 24
    Kay says:

    @mb:

    Not so fast, Leftists! Horrible Right wing lawyer Koback wants a word with the Democrat:

    “In another huge twist in the Kansas Senate race, Kansas Secretary of State Kris Kobach (R) said Thursday that Democratic Senate candidate Chad Taylor must remain on the ballot despite his declaration that he would withdraw from the race.
    According to a statement published by the Washington Post, Taylor said he had consulted with Kobach’s office in drafting his paperwork to withdraw his name from the ballot. Taylor said he had spoken specifically with Brad Bryant, director of elections and legislative matters in the secretary of state’s office.
    “I again confirmed with Mr. Bryant that this notarized letter removed my name from the ballot,” Taylor said. “He again said ‘Yes.'”

    Koback is a voter fraud fraudster and he’s none too fond of Latinos. He was Mitt Romney’s immigration advisor until Romney (pretended) to disassociate himself because he’s controversial, which Romney probably didn’t know because it’s Kansas :)

  25. 25
    Enhanced Voting Techniques says:

    @jl: It’s Kansas, they would never vote Dem for tribal reasons. But they will vote for an independent. That’s why Kobach is desperately trying to negate it.

    Speaking as a former conservative you don’t walk away from it over night, it takes baby steps.

  26. 26
    Marcelo says:

    @FlipYrWhig: I think what you need is the “Tea Party” of the Left, a really charismatic and successful uprising of sorts that’s still within the party, that ends up moving the party to the left. Occupy did this to some extent, but you know, herding cats and all. Plus we don’t have Fox News carrying our water.

    The general rise of Internet feminism/LGBTQ/Twitter mocking people like Jamie Dimon/Robin Thicke is not bad because it’s so commonsense. There’s just no way to centralize it the way grifters did with the Tea Party.

    But no, a new party isn’t the answer. It has to be an intraparty push to the left.

  27. 27
    Bobby Thomson says:

    @Richard Mayhew:

    But the composition of the off-year electorate should (especially by now) be baked into the polls of likely voters

    I disagree. I think it’s too early to determine who likely voters really are. A lot of pollsters don’t even ask those questions this far out. Time will tell if Democrats come back strong from their annual August slump, whether they revert to off-year form, and whether the story they tell pollsters in October about their likelihood of voting is true.

  28. 28
    Bobby Thomson says:

    @Kay: As much as I detest Kobach, his argument isn’t frivolous. The statute says (with my added emphasis):

    Any person who has been nominated by any means for any national, state, county or township office who declares that they are incapable of fulfilling the duties of office if elected may cause such person’s name to be withdrawn from nomination by a request in writing, signed by the person and acknowledged before an officer qualified to take acknowledgments of deeds. Any such request shall be filed with the secretary of state in the case of national and state offices and with the county election officer in the case of county and township offices.

    Taylor didn’t expressly state that he would be incapable of fulfilling the duties of the office, and my understanding is he’s keeping his options for future elections open. Perot was able to switch his VP candidate without complaint, but the statute was amended to add this language after that.

    @Enhanced Voting Techniques:

    It’s Kansas, they would never vote Dem for tribal reasons. But they will vote for an independent. That’s why Kobach is desperately trying to negate it.

    No, that doesn’t make any sense. If people would never vote for a Democrat all of this would be irrelevant. People would, do, and have voted for Democrats for state wide office. Just ask Governors Docking and Sebelius. Kobach isn’t knocking the independent off the ballot. He’s keeping the spoiler on it.

  29. 29
    Robert Waldmann says:

    I wish I was convinced but note that WaPo NYT and 538 use a mixture of type1 models and polls not pure fundamentals based models. The fact that Democrats outperform the fundamentals does not mean that they outperform the mixed models. The fact that models are more complicated with more variables does not imply that the added variables are “confounding”

    The key word is “all” in “The pure poll aggregators basically say that *all* of the fundamentals will be baked into the polls by the summer before the election, ” Importantly, the fact that you got called all Senate races correctly in late October 2012 doesn’t mean the polls only approach works in early September.

    I called 3 wrong in 2012 (ooops). I called 0 wrong in Summer 2010 (must have been luck). My approach was to look at polls and then add a batshit insane correction (as in Ken Buck and Sharron Angle are batshit insane and a few people who now plan to vote for them will notice before election day– O’Donnell is also nuts but was not ahead in the polls).

    After my experience in 2012, I won’t even discuss applying this procedure to 2014 and will in no way suggest that voting to raise the Medicare age to 70 might affect a candidate’s chances.

  30. 30
    Panurge says:

    @John M. Burt:

    The Greens will have to figure out how to build a party first. Americans seem to have no idea how it’s done.

    The mistake the Dems made was abandoning the left in the late ’80s to go for the center, figuring the left had nowhere else to go. I’d rather have a country with a broad coalition just right of center and a steadfast left-wing party than a broad coalition just left of center and a steadfast right-wing party. At least under the first arrangement left-wing ideas get heard–and eventually they win, if only due to voters wanting a change. Going for the center is fine, but the point is to build a coalition, not just to drop one set of voters for a bigger set–especially if the bigger set still isn’t a majority.

  31. 31
    Panurge says:

    @Marcelo:

    There’s one problem with that, which is how it worked out when they tried that in 1972. That’s why the Dem establishment is so wary of such things. If they’d stood fast, the Dems might be in a better place now, but they freaked out (being, in the end, an Establishment party). Now what?

  32. 32
    Panurge says:

    @mb: If the end result is less Republicans, too, what does that mean, then?

    BTW: “Art of the possible“, not the impossible.

Comments are closed.