Compared to what?

I recently took my college referee fitness test.    The college fitness test has four events; a distance run, an agility box, and two sprint sets.  On each event, the referee can score between zero and five points depending on time/distance.  I passed.

My results could be described in the following ways:

  • Piss Poor
  • Adequate
  • Great

These disparate descriptions are accurate.

They are accurate depending on what my results were compared against.

My results were piss-poor when compared against the other nine referees in my testing group.  It was me, seven referees whose highest level matches last year were either NCAA semi-finals, or some version of an international match, and two 21 year olds who run the 800 in college.   In all four events, I had or shared the lowest score.  Against a narrow comparison group, I was slower than molasses. 

My results were adequate in that I am fast and fit enough for the games that I work.  I met or exceeded external absolute standards.

My results were great because I had personal bests in two events (the distance run and the first set of sprints) and matched my previous personal best scores on the other two events.  Against an internal comparison, I improved. 

Asking the simple question of what something is being compared against and therefore being evaluated against is a critical question in determining the value of an analysis. 

Moving back to the healthcare world, this simple question — compared to what — is critical when looking at the Exchange age mixture.

Jonathon Cohn at the New Republic has a good summary:

And within those marketplaces that the federal government is managing directly, 28 percent of enrollees are ages 18 to 34…

As for the age mix, you may have heard that about 40 percent of the population eligible for coverage in the marketplaces is between the ages of 18 and 34. That’s true and, obviously, 28 percent is a lot less than 40 percent. The worry has always been that older and sicker people would sign up in unusually high numbers, forcing insurers to raise their prices next year and beyond.

But insurance companies didn’t expect young people to sign up in proportion to their numbers in the population. They knew participation would be a bit lower and they set premiums accordingly. Only company officials know exactly what they were projecting—that’s proprietary information—but one good metric is the signup rate in Massachusetts, in 2007, when that state had open enrollment for its version of the same reforms. According to information provided by Jonathan Gruber, the MIT economist and reform architect, 28.3 percent of Massachusetts enrollees were ages 19 to 34, a comparable age group.

Age is a good but not perfect proxy for health.  The Exchanges need a population that is roughly similar in health composition to the general population to avoid premium spikes. An Exchange population of only Balloon-Juice blog hosts or 63 year olds with chronic conditions is an extremely expensive risk pool.  A death spiral would be unlikely due to subsidy design but total federal costs would increase dramatically.  An Exchange population with a good number of healthy people in it means premiums won’t spike. 

So when you hear people comment on whether or not the Exchange risk pool composition is good or bad or adequate, ask — compared to what?

Against the most relevant comparison (Massachusetts in 2006), the risk pool composition is at least adequate if not good on a national scale.  We don’t have the data to say what the Exchange risk pool looks like in any given state.  My suspician is that states that embraced PPACA, the risk pools will be on average, better than the full resistance states.  The states that embraced Obamacare have seen higher sign-ups and more effective outreach efforts.  Young and healthy individuals have always been the hardest group to bring en masse into social insurance programs, so states that actively outreached to these groups probably would see higher enrollment than states like Georgia that actively ratfucked outreach efforts.

The first year risk pool age composition is not meeting ideals, but a reasonable expectation is that it would never reach ideal composition in the first year. 

19 replies
  1. 1
    Jerzy Russian says:

    Nice post. I once had a (college) soccer player in class. I asked him how far he ran during a typical game and he estimated 4 to 6 miles, often times at a fast pace. Amazing.

  2. 2
    Richard Mayhew says:

    @Jerzy Russian: Center referees on high level games will run six miles or so including 1,000 meters or so of very high speed sprints. ARs will run 4 to 5 miles with a higher proportion of high speed sprints.

    Midfielders for European professional teams will get five to eight miles in a game. Forwards tend to get less distance but more balls to the walls sprints.

  3. 3
    Amir Khalid says:

    @Richard Mayhew:
    Since you did pass the referee fitness test, you shouldn’t rate yourself below adequate. Until Randinho comes back — and I’m starting to despair that Real Life (TM) will ever let him be free enough to do so — is there any chance you could fill in as our football front-pager?

  4. 4
    cmorenc says:

    I’m a USSF 7 soccer ref who’s about to turn 65 yo in a month, and required to take an annual fitness test to maintain grade. The USSF grade 7 test (different than the NCAA test) is comprised of three elements which I could have passed in my 20s in street shoes while carrying a brick in each hand: 1) run continuously for 12 minutes while covering at least 1800 meters in that time (5 laps around a standard track (note: it’s 2000 meters if you’re under 45yo); 2) run a 50 meter sprint within 9.5 seconds (9.25 if you’re under 45); 3) run a 200 meter sprint within 40 seconds (all ages). The first two still mostly represent a chore more than a challenge to me, but the 200 meter sprint is in my personal nightmare closet as each annual re-test approaches. It requires maintaining a nearly all-out sprint pace over that distance, when the body is really only up for the first 50 before a strong urge to cut back to a fast jogging pace kicks in, and making the requisite time requires the ability to summon a huge rush of willpower to maintain the pace the remaining 150 meters. When the time comes when I’m either unable to pass this simple fitness test, or unwilling to undertake the extra bit of training to prepare for it, then it will be time to quit soccer refereeing. Alas, I think that time is approaching. I may have it in me to pass it one more time, but no more after that. I’ll either retire at the end of 2014 (if I don’t take the test this summer) or else at the end of either spring or fall season in 2015. I enjoy doing competitive-level games enough that I’m not going to cut back to doing easy rec-level games.

  5. 5
    Richard Mayhew says:

    @Amir Khalid: No, I see the game as a referee, not as a player, not as a coach and definately not as a fan. I don’t follow the European leagues enough to know most of the high level players and their styles/preferences. I’ll talk about controversial or strange calls from my point of view as a referee, but I can’t bring the depth of knowledge and passion as a fan that Randinho can bring to the issue.

    As a referee watching a game from the sidelines/TV, I probably only watch the ball for a third of the time. My eyes are on the next play probability space(s) or seeing how the center is working with his ARs to box play/control idiots/avoid problems.

  6. 6
    wvng says:

    “We don’t have the data to say what the Exchange risk pool looks like in any given state.” I am waiting to see what happens in WV and to our rates. Signup through the exchange was week, but I’m sure BCBS had already assumed an old and sickly population so maybe things won’t change much. Wish we had a national risk pool so us old/sick states could get buffered by healthy young states.

    Excited to hear we will have at least one more player in the exchange next year.

  7. 7
    ThresherK says:

    @Jerzy Russian: Yeah, and it’s not even the running.

    Whenever I go from my best sport (volleyball) to soccer, there’s a big increase in the twisting, turning, and lurching required. Helloooo, shin splints. (I’m sure basketball is the same way–too much quick unpredictabiliy for my ankles and long strides.)

    I would like to know if any officials are tested on kinetic vision during tests and drills, especially the box agility.

  8. 8
    cmorenc says:

    @Richard Mayhew:

    As a referee watching a game from the sidelines/TV, I probably only watch the ball for a third of the time. My eyes are on the next play probability space(s) or seeing how the center is working with his ARs to box play/control idiots/avoid problems.

    When watching a game as a spectator, my soccer-ref background often induces me to look for both the developing play and where the CR (center referee) is located. The good ones are constantly moving and adjusting to blend with the momentary flow of the game, always striving to attain balance between close-enough to convincingly see and assert awareness of their presence among the players, versus not interfering with spaces needed by players or developing play. Play in the “coffin corners” (quadrants opposite the assistant referee in each respective end) are a particular challenge for the center ref – the general idea is for the CR to try, to the extent feasible, to maintain both play and one of the ARs within his field of view, but as play moves toward either of the “coffin corners” it can be challenging to achieve this goal without falling into the problem of moving into an increasingly “extreme” position (meaning in this context, one from which a reversal of direction of play can quickly leave the CR problematically distant from developing play, requiring a long sprint to get back into adequate position to see or easily “sell” any call that might need to be made). Soccer refereeing, to be done adequately, let alone well, requires a significantly higher level of fitness by both center referees and assistant referees than any other sport.

  9. 9
    Mnemosyne says:

    Age is a good but not perfect proxy for health.

    I would be very curious to see if there are any studies on whether it really makes a financial difference to treat chronic conditions from a young age. Intuitively, it would make sense to me that treating a 25-year-old for (say) depression and having ongoing treatment would cost X amount to the insurer, but it would be made up in the overall economy by making that person more productive and able to keep a job, get promotions, etc.

  10. 10
    Richard Mayhew says:

    @Mnemosyne: I’m not sure… the dominant factor is that a young person is just far less likely to need long term continous maitenance care for anything, and if they do, they tend to heal faster on the physical health side.

    Oh to be 22 again or at least to have my knees at 22 instead of my current ones……

  11. 11
    Richard Mayhew says:

    @cmorenc: I would not want to try hockey refereeing as that is a full body work-out as well with numerous all-out sprints.

    Agreed with you — the coffin corners are what kills referees who aren’t fit enough. They either get too deep to compensate for their lack of speed, get in the way and get caught on the big ball counter-attack OR they are aware of their lack of speed/fitness and hover 30 yards off the ball and hope to see enough to assert control. That covers their asses for the counter-attack but not the hand fighting in the corner.

  12. 12
    imonlylurking says:

    I’m sorry, Richard-I can’t trust anything you write until you DISCLOSE YOUR INCOME!

    /bored today

  13. 13
    TG Chicago says:

    Obligatory theme song for this post:

    “Compared to What?” — Roberta Flack

    Possession is the motivation
    that is hangin’ up the God-damn nation
    Looks like we always end up in a rut
    Tryin’ to make it real — compared to what?

  14. 14
    Anoniminous says:

    when you hear people comment on whether or not the Exchange risk pool composition is good or bad or adequate, ask — compared to what?

    The question should be asked for all comparative analysis, quantitative or qualitative, along with its compadres: How Was It Done? So What? and Who Cares?

    Two necessary monographs to cut through the crap:

    How to Lie with Statistics

    On Bullshit

  15. 15
    cmorenc says:

    @Richard Mayhew:

    @cmorenc: I would not want to try hockey refereeing as that is a full body work-out as well with numerous all-out sprints.

    You’re right – I neglected to consider ice hockey as a sport requiring not merely a comparable level of fitness to soccer but skating skills on top of frequent hard sprints. (the constant sprinting and fast-movement of the puck more than counterbalances the much smaller playing space they need to cover than a soccer field). An overweight or less-than-fit referee in competitive hockey is more in over their head than even they are in soccer.

  16. 16
    grrljock says:

    “Compared to what?”–also the central question that Edward Tufte always asks. I find your posts on this topic (health policy/insurance, not refereeing, sorry) really interesting. Please keep posting them!

  17. 17
    Richard Mayhew says:


    I find your posts on this topic (health policy/insurance, not refereeing, sorry) really interesting

    Then I’m going to take my ball and go home :)

    Actually, I figure if John can talk about mustard, field cars and cooking, I can talk about one of my passions too… and enough people will stick around for the high value content.

  18. 18
    grrljock says:

    @Richard Mayhew: I didn’t mean to sound so harsh. I appreciate fair and fit referees! Some of my good friends are rugby referees.

  19. 19
    Richard Mayhew says:

    @grrljock: so they are crazed lunatics — I get to see a couple of rugby matches a week as one of the major fields around here has half a dozen soccer fields and a pair of rugby fields. Those rubgy fields are booked solid on Saturdays, and it is a fun thing to watch a sport I have no idea what the referee is doing/seeing. The biggest difference I have noticed is that the only time a player speaks to the referee is to inform him where the post-match beer is located.

Comments are closed.