 |
||||||||
Organizational Effectiveness Through Feedback Management
|
||||||||
>> Sears IVR Customer Satisfaction Survey
|
||||||||
Review our
Read our latest article on
and learn the dramatic shortcomings in the survey design
|
|
Learn how to conduct a survey project from questionnaire design
to survey administration
to data analysis.
 August 7-9, Las VegasOctober 28-30, Chicago December 9-11, BostonLate February, Phoenix~~
View a short online presentation
summarizing the concept of surveying and outlining our courses.
|
Provides targeted assistance on your survey questionnaire and project while being mentored on good survey practices.
|
Explore next-generation ideas in customer satisfaction &
customer interaction at our
October 7 at the world famous,
The International Golf Club, Bolton, Mass.
|
Need more than
targeted survey consulting?
Great Brook specializes in
the design and execution of
Custom Survey Research Projects to meet
your individual needs.
|
Get an experti's review of the feedback management processes in your organization for customers or other stakeholder groups --
or get a deep dive assessment of your survey program.
|
|
Have a question about surveying you'd like to have answered in a future Insight article?
Please contact us.
|
Self-help guidance for your survey project
 |
Private On-site Survey Workshops are available at your company. This may prove more cost effective and deliver immediate value to your survey program as we build your survey instrument in class.
Please contact us for details.
|
For a comprehensive list
of survey automation tools
for your survey program.
|
You would probably think that mistakes in survey design would be made by small companies with limited resources and knowledge. Yet, some of the best examples of bad survey design practice I find in big companies. In this article, I'll illustrate the mistakes made in the IVR survey to measure Sears customer satisfaction. (Home Depot's store visit survey is another glaring example of bad survey design.) Read my related article regarding to learn the strengths and weaknesses of IVR survey admnistration method.
In a nutshell, the Sears IVR customer satisfaction survey:
Didn't allow me, the customer, to fully express my feelings about my entire transaction with SearsForced me to answer irrelevant (to me) survey questions Force fit a suvey question to a scale to the point where the data generated are not interpretable (that is, they generate garbage data)Asked me for comments -- and then ignored me.Most importantly, lead to greater customer dissatisfaction and destroyed any customer loyalty I had to SearsThe last bullet should startle you. Customer surveys are typically meant to provide customer measurements to increase customer satisfaction. So, how could a survey program design actually lead to greater customer dissatisfaction? The answer lies in a poorly conceived survey project and program design that leads the customer to come away with a more negative attitude toward the company in addition to generating survey data of questionable value.
Before I explain all of these survey design problems in detail and how to avoid them -- hint: the answer lies in having a good customer survey program design -- some background on my interaction with Sears customer service is needed.
My Sears Service Interaction
I bought a hot water heater in 1997 from Sears. It had a lifetime warranty. In September of 2007 the heater failed; it cracked. My transaction with Sears evolved into the following series of interactions.
1. Claim Initialization. I called the Sears call center at 800-4MyHome to inquire how I would invoke my warranty protection for the failed hot water heater. This interaction was great. They actually had my purchase on record, so I didn't even need my receipt, which I did have. They arranged for a repair tech to come to my house two days later to verify whether the tank failure was covered under warranty. I wasn't thrilled with the 2-day wait, but I had backup hot water capabilities. The agent told me that should the technician verify that the failure was covered by my warranty, then I would not be levied the $69 service charge for the technician visit. The explanation was poorly worded, so I restated what I thought the agent said to get verification.
2. Technician Verification of My Claim. The technician came to my house, verified my claim, gave me the paperwork to get a new hot water heater at my local Sears store - and charged me the $69 service charge. Foolishly, I paid it (with my credit card). I figured getting my refund from a company of Sears' stature would be no problem. Boy was I wrong.
3. Fulfillment of my claim. The field tech gave me a form to take to my local store to get my replacement heater. My day was shot so I went to the Marlboro Sears store to get my new heater, but they told me the paperwork would take 24 hours to get into the IT system. Why didn't the technician tell me this? I was told I would get a call the next day. I didn't. I had to call them. I did eventually get my new heater, got it installed, etc.
4. Resolving the Improper Charge. When questioning the charge with the field tech at my house, he told me to call the local office in Danvers, Mass. rather than call the service center. He said I could talk to a “customer relationship specialist” there. The person with whom I spoke was the most useless, aggravating person I have ever engaged. He cut me off in mid sentence, refused to let me explain the situation and basically told me whatever the tech said is right. He was completely unconcerned about the discrepancy between the service center and the tech. He was so exasperating, I hung up on him. If he is a “customer relationship specialist,” then Sears is in big trouble.
I called 1-800-4MyHome again and verified that I should not have been charged. They told me to get my refund by working through my local Sears store. I tried unsuccessfully for months. I wrote to the Sears headquarters about this likely illegal practice in the hopes I could avoid going through my state's consumer affairs division.
A month after my letter was delivered I got a call from a troubleshooter at Sears headquarters. She told me the charge was legitimate. The service agents at 800-4MyHome gave me wrong information and Sears is not responsible if its agents give a customer wrong information that is the basis for establishing a contractual arrangement between Sears and the customer. Tough luck. End of story. I was flabbergasted. My jaw dropped.
|
Soon after the technician's visit, I received an automated phone message asking me to take an IVR survey. I should have transcribed the survey, but I can do it good justice. The 6 or 7 question survey -- it was one question longer than stated in the introduction -- posed all its questions on a satisfaction scale, where 1 represented Very Dissatisfied and 5 represented Very Satisfied. All of the questions pertained to the technician's visit to my home, which, I'll argue, was a glaring mistake. But the biggest mistake was what Sears did with my survey data.
What Lessons Can We Learn from the Sears IVR Survey?
Keep the Survey Design Focused. I teach in my Survey Design Workshop to keep a tight focus on the survey contents and not let other departments meddle their way into the survey design process. This I believe happened at Sears. The first 6 questions were about the technician's performance, but the last question, asked about my “satisfaction with the technician making me aware of Sears' products and services.” That's not something about which I give two hoots, and I suspect the operational folks in Sears service don't either. But the marketing folks do. Clearly, the technician is tasked to promote Sears products, which is one of those strange -- and inappropriate roles -- for service technicians. Survey focus is achieved by having a good Statement of Purpose or Research Objectives -- and sticking to it.
Word Questions to the Scale. The question mentioned immediately above also displays a classic problem in survey question design: wording a question to fit the scale. Look at the question. It was posed on a 1 to 5 satisfaction scale. I had no idea how to respond. The technician didn't make me aware of any Sears products. I didn't know he was supposed to, and I was glad he didn't. So, does that mean I should have scored it a 5 indicating Very Satisfied with the technician, but not because he nicely made me aware of Sears products but because he didn't waste my time with a sales pitch that I would at best find annoying? Or should I score it a 1 indicating Very Dissatisfied because... well, I don't know why I would. But I am sure the marketing folks who pushed for this question would want me to score it a 1. I didn't. I did the cop-out, middle-of-the-road, 3. Regardless, the marketing folks at Sears are interpreting the results from this survey question that is simply generating garbage data.
Develop Comprehensive Response Options. That last question really needed a Not Applicable option. I tried just not answering, but the system would not let me do that. So, as said, I entered a 3. I had a sense I would soon get the opportunity to answer an open-ended question, so I had to get beyond this question.
Develop Comprehensive Question Set. While we want to keep any survey, but especially an IVR transactional survey, short and sweet, you still need to pose all the needed questions. Usually, my criticism of surveys is that they go too far a field. The Sears survey was too narrow. The technician gave me factually incomplete information about how to get my replacement heater, and I was charged improperly. Those critical flaws in the transaction went unaddressed in the survey questions. Trust me; those factors have made the deep impressions on me about Sears on-site service.
Think Beyond the Interaction to the Transaction. A service transaction is a chain of interactions, and the weak link needs to be identified. The Sears survey only asked me about my experience with the technician. My transaction with Sears went well beyond the technician's visit, including my interaction with the service center and the store. No one ever asked me about those interactions, and thus did not allow me to express my full feelings about my experience. (You're probably thinking, “but wasn't there an open-ended comment question?” Hold that thought!) Yet, I'm sure those who designed this awful survey think it's great. I, as the customer, came away very frustrated since I had had a poor experience and did not get to say my piece.
Notice how this issue relates to the need for a good Statement of Purpose. While the typical customer (survey respondent) won't think in these terms, they will know if a survey is comprehensive and well designed. The Sears IVR survey wasn't. Why? My guess is that Sears has very stovepipe functional groups that do not interact.
Need for a Service Recovery Program. The above mistakes are bad but not egregious. The most disastrous error in the Sears survey is what they did NOT do with my data. The last question presented the opportunity to make a free-form comment about my experience. I was waiting for that opportunity. I even developed notes to be sure I covered everything. I listed the litany of mistakes that Sears made. You would think that the person who analyzes that textual data would flag me for a follow-up call. A well-run company with a well-designed survey program would have done that. Sears did not. They ignored my cry for help, making me angrier than I already was. Yes, the survey -- designed to measure customer satisfaction and identify needed operational improvements -- actually made the situation worse!
Need for an Integrated Feedback Management Strategy. Even if someone had read and responded to my comments, it might not have mattered. I recounted in the background on my Sears transaction that Sears headquarters basically told me to get lost. Sears sees no ethical or legal obligation to honor the verbal contract established by its service agents. This is a pretty startling indictment of the strategic role of customer service at Sears. But why does Sears go to the bother and expense of having this one feedback survey that sits like an isolated island disconnected from its potential business value?
Will I ever shop at Sears again? You know the answer. That $69 savings is quite the Pyrrhic victory for Sears.
Addendum: Missed Value in a Service Recovery Act. After the headquarters' troubleshooter told me to get lost, I started looking at my legal options, including initiating a greviance through my state's attorney general office. Then, 6 weeks after phone call with Sears, a check for $69 shows up from Sears. No accompanying letter. No accompanying phone call.
Part of the purpose of a service recovery act is to win back the hearts and minds of the customer. The check partially won back my mind, but my heart? I'm still seething at the process and how I was treated. Compare this to how Subaru handled its service recovery of a flawed service recovery. Sears missed an opportunity here to win me back.
|
-- Fred Van Bennekom, Dr.B.A., Principal, Great Brook
Please contact us with feedback on this article or other topics that you might find interesting.
Add your name to our Service Insights newsletter which highlights newly published articles.