Workshop Registration                 Order Form                 Site Search
Organizational Effectiveness Through Feedback Management

 >>  Money Grows on Trees -- If You Believe the Polls      

in Excel -- free.
Also use it to calculate statistical accuracy based on the response group size.
Review our
Synopsis of Articles
Read our latest article on
and its shortcomings
Survey Workshops
Learn how to conduct a survey project from questionnaire design
to survey administration
to data analysis.

Next Workshops
June 23-25, Dallas, TX
Sept. 16-18, Wash. DC
Oct. 27-29, Orlando (tent.)
Early Dec. TBD, Dubai UAE
~~~
Offering TBD
~~~
View a short online presentation
summarizing the concept of surveying and outlining our courses.
Receive our Newsletter
Email:
Survey Products
Dr. Fred Van Bennekom
conducted a webinar
in conjunction with GainSight
on March 30, 2013 on the
Hidden Dangers
of Survey Bias
Private On-site
Survey Workshops
are available at your company.  This may prove more cost effective and deliver immediate value to your survey program as we build your survey instrument in class.  
Please contact us for details.  
Survey Mentor Suite
Provides targeted assistance on your survey questionnaire and project while being mentored on good survey practices.  
Custom Survey Projects
Need more than
targeted survey consulting?  
Great Brook specializes in
the design and execution of
Custom Survey Research Projects to meet
your individual needs.
Voice of the Customer Audits
Get an expert's review of the feedback management processes in your organization for customers or other stakeholder groups --
or get a deep dive assessment of your survey program.
Service Strategy
Have a question about surveying you'd like to have answered in a future newsletter article?  
Please contact us.  
Self-help guidance for your survey project
Survey Tools
For a comprehensive list
of survey automation tools
for your survey program.
Summary: Political polls -- as well as organizational surveys -- many times present conflicting results within a poll. The reason is that the surveys have not been designed to force respondents to engage in trade-offs among conflicting options. We see this in the New York Times, CBS News poll of swing states released on August 23, 2012 where the poll indicates that respondents want to keep Medicare as we know it yet spend less on it. Clearly, something is amiss.

~ ~ ~

The NY Times, CBS News and Quinnipiac University released a poll of swing states (FL, OH, WI) on August 23, 2012. The key finding from the headline, "Obama Is Given Trust Over Medicare," was summarized as: 

Roughly 6 in 10 likely voters in each state want Medicare to continue providing health insurance to older Americans the way it does today; fewer than a third of those polled said Medicare should be changed in the future to a system in which the government gives the elderly fixed amounts of money to buy health insurance or Medicare insurance, as Mr. Romney has proposed. And Medicare is widely seen as a good value: about three-quarters of the likely voters in each state said the benefits of Medicare are worth the cost to taxpayers. 

But here's the question posed from the details of the survey results, which thankfully the NY Times does publish:

35. Which of these two descriptions comes closer to your view of what Medicare should look like for people who are now under 55 who would be eligible for Medicare coverage in about ten years? Medicare should continue as it is today, with the government providing seniors with health insurance, OR, Medicare should be changed to a system in which the government would provide seniors with a fixed amount of money toward purchasing private health insurance or Medicare insurance. (Answer choices rotated) 

Just over 60% wanted to continue Medicare as is, and about 30% said they supported changing the system.  

Now, look at the results for the next question:

36. To reduce the federal budget deficit, would you support major reductions, minor reductions, or no reductions to spending on Medicare? 
 
Almost 60% of respondents supported major or minor reductions in Medicare (roughly 11% Major, 48% Minor).  

The Times inexplicably doesn't report this latter finding from their survey. In fact, the headline for the article could easily have been, "Strong Majority Favor Reductions in Medicare Spending."

But how can 60% support keeping Medicare as is yet the same percentage support spending reductions?  The survey design did not force respondents to make trade-offs among competing alternatives, and these conflicting results show why forcing respondents to make trade-offs is so important.  Forced trade-offs eliminate the money-grows-on-trees responses we see here. When reviewing poll findings, I frequently find such conflicting results -- and only selected results are reported in the write-up.

Perhaps more puzzling is that the question as phrased is not grounded in how normal people think, that is, people who live outside of the Washington DC beltway. No one is proposing that Medicare spending should be reduced.  At issue, is the rate of growth in Medicare spending. David Wessell of the Wall Street Journal in summarizing the Congressional Budget office analsysis says that Ryan is proposing a Medicare be 3.5% of our Gross Domestic Product (GDP), which is the total output of our economy, in 10 years versus  4% of GDP if the program stays as is.  Currently, Medicare consumes 3.25% of GDP.  With the expected growth in GDP, even under a Ryan plan Medicare spending is increasing.

Reducing spending on Medicare could be interpreted as:
Reducing per capita spending on each Medicare recipient
Reducing the overall spending on Medicare, that is, the total spent each year
Reducing Medicare spending as a percentage of GDP
and maybe some I'm not thinking of!

How did you interpret the phrasing in Question 36? Since the leading phrase in the question was "to reduce the federal budget deficit" my educated guess is that the second option above is what most people were thinking. That's the only option that would actually "reduce" the deficit -- as opposed to slowing the growth of the deficit.

Regardless, with such ambiguous phrasing, it's near impossible to interpret the results except that 60% support some kind of reduction, a position that is incompatible with keeping Medicare "as it is today."

My conclusion is that this phrasing shows how rooted the poll designers are in Washingtonian logic.  Only in Washington is a slowing of growth rates in spending, even on a per capita basis, considered a "reduction."  Imagine the polling results if they had presented it accurately.

~ ~ ~

Another interesting element in the questionnaire design can be found in the the question immediately preceding question the Medicare change question:

34. Overall, do you think the benefits from Medicare are worth the cost of the program for taxpayers, or are they not worth the cost? 

The poll found roughly consistent results for the three states with 75%-16% feeling that Medicare is worth the cost. That question helps set the mental state of the respondent that Medicare as we know it is a good thing going into the next question about making changes to the program.

We should also note that Question 35, does not present the proper choices to the respondent. Congressman Ryan's 2011 plan did call for offering only premium support to those currently under 55 when they reach Medicare eligibility. However, the 2012 Ryan plan offers the choice of premium support or staying in traditional Medicare. In other words, the poll did not test the actual choice offered between the two campaigns even though that is how the Times has pitched the results of the poll.  

Further, while the headline is that "Obama Is Given Trust Over Medicare," the poll has mixed results.  While by a  51%-42% margin Obama is trusted more to handle Medicare, more people strongly disapprove of ObamaCare than strongly approve. 

Perhaps the most startling result in the poll -- and not reported by the Times -- was the seismic shift in the Florida senatorial race.  In the Time's late July poll, Democrat Bill Nelson led Republican Connie Mack 47%-40% while in this poll, Mack led 50%-41%.

                                                                                                                                                                                                                                                                                                           
Great Brook Home   |   Products & Services   |   Events   |   Clients   |   Partners   |   Resources   |   News   |   About Great Brook   |   Contact Us   |   Site Search & Map
Copyright Great Brook Consulting, 2000-2104.  All rights reserved.   Privacy Policy   Site Map