 |
||||||||
Organizational Effectiveness Through Feedback Management
|
||||||||
|
>> Delusions of Knowledge
|
||||||||
Review our
Read our latest article on
and learn the dramatic shortcomings of a pooled service delivery design -- and horrific Servcie Recovery.
|
|
Learn how to conduct a survey project from questionnaire design
to survey administration
to data analysis.
 March 2-4, 2010 DubaiMarch 15-19, SingaporeMarch 23-24, Kuala LumpurApril 21-23 2010, Wash DCJune 21-23, 2010, Chicago~~~
TBA Summer/Fall 2010~~~
Learn how to design a
~~~
View a short online presentation
summarizing the concept of surveying and outlining our courses.
|
Provides targeted assistance on your survey questionnaire and project while being mentored on good survey practices.
|
 Contract Holder GS-02F-0126U
|
Need more than
targeted survey consulting?
Great Brook specializes in
the design and execution of
Custom Survey Research Projects to meet
your individual needs.
|
Get an experti's review of the feedback management processes in your organization for customers or other stakeholder groups --
or get a deep dive assessment of your survey program.
|
|
Have a question about surveying you'd like to have answered in a future Insight article?
Please contact us.
|
Self-help guidance for your survey project
 |
Private On-site Survey Workshops are available at your company. This may prove more cost effective and deliver immediate value to your survey program as we build your survey instrument in class.
Please contact us for details.
|
For a comprehensive list
of survey automation tools
for your survey program.
|
The Dangers of Poorly Done Research
Imagine you're planning to conduct a survey to support some key business decision-making process. But you have to get results fast. So, you throw together a quick survey since something is better than nothing. Right? Wrong.
This article is from our Service Insights Newsletter.
|
Why is this wrong? Because this knee-jerk survey process may contain biases and errors that lead to incorrect and misleading data. Thus, the decisions made from this data will be based on delusions of knowledge. The purpose of this article is to outline the types of biases and errors common to survey projects. Knowledge of these will help you create a survey that provides meaningful results.
First, though, let's drive home this issue of true knowledge versus perception. Consider the following matrix. (As a quasi-academic, it's a requirement to present ideas in 2x2 matrices. <grin>)
|
|
Our True Knowledge of
Objective Reality
|
|
|
|
We Do Understand
|
We Do Not Understand
|
Our Perception of Our Knowledge of Reality
|
We Think We Understand
|
Nirvana
|
Delusions of Knowledge
|
We Think We Do Not Understand
|
Area of Opportunity
|
No Loss
|
|
In this matrix, our true understanding of objective reality - we know the truth or in fact we don't - is contrasted to our perception of our understanding of reality - we think we know the truth or we think we don't. Let's look at each quadrant.
Northwest Quadrant: We think we know reality and in fact we do. This is our goal. We want to live in this cell. Nothing but blue skies do we see…
Southwest Quadrant: We think we don't know reality but in fact we do. This is a good zone because of the opportunity it presents. We're not going to run off half-cocked and do something stupid. We're cautious in this position awaiting someone - a manager, a mentor, our mate - to tell us that we know more than we thought and to give ourselves more credit. In the Knowledge Management field this is known as Tacit Knowledge in contrast to Explicit Knowledge found in the NW quadrant.
Southeast Quadrant: We think we don't know reality and in fact we don't. This is another good zone since we're unlikely to make wrong decisions. However, that opportunity in the previous quadrant is lacking since we lack real knowledge. Our mentor first has to help us learn, and then realize what we have learned. We want to move to the Southwest quadrant and not to the Northeast quadrant.
Northeast Quadrant: We think we know reality but in fact we don't. This is our black hole of decision making where we're operating under delusions of knowledge. Many versions of this exist. Maybe one customer or employee complains and the assumption is made that this is a real problem that everyone confronts. Maybe we've done some “research” but the research plan or execution is flawed. A whole host of reasons can create delusions of knowledge. You probably know many people who fit this quadrant: “He doesn't know what he's talking about.”
But how could a survey research project create such delusions? “A survey is so simple,” you say. “I get surveys in the mail all the time, and I've thrown together my own surveys. All research is beneficial. How could any survey effort not help? This isn't brain surgery, after all.” But it is (sort of). We are trying to get into the brain of some group of interest -- our customers, employees, stockholders, suppliers, etc. - and measure their perception of us and perhaps collect some other hard data.
A survey project presents many, many opportunities to screw up. The following table lists the areas where biases or errors can affect our survey results - and thus our decisions.
Stage of Survey Project
|
||
Questionnaire Design
|
Survey Administration
|
Data Analysis
|
Instrumentation bias (bias from question wording, etc.)
|
Incorrect sampling procedure
|
Bad model for analysis
|
Questions that do not support desired analysis
|
Improperly defining the relevant population
|
Error in analysis
|
|
Administration technique bias
|
Weak presentation of the findings
|
Response bias
|
Non-response bias
|
|
In any research program, including surveying, the researcher faces a constant struggle to avoid introducing biases into the data and errors into the findings and conclusions. For a novice surveyor real danger exists. Why? Because the novice isn't even aware that these biases could exist, they don't know enough to avoid them, they won't know they've created these biases, and they won't know to take caution when interpreting the collected data set. The novice will execute the flawed project, analyze the data and draw conclusions using data that doesn't represent the true feelings of our survey population, our group of interest.
Future articles will address each of those three major areas in depth. I'll add links when they're written. The bottom line message is that rigor is needed in any research project to be sure that you're truly learning something about the area of interest.
In summary, would you rather make decisions based on:
1) Sound research data, or
2) Data you thought was right but was wrong or
3) Gut feel and business intuition?
While I would always prefer the first choice, I'd rather operate from intuition than delusions of knowledge. As some wise folks over the millennia have stated in various forms:
''The trouble with most folks isn't so much their ignorance. It's know'n so many things that ain't so.'' - 19th century humorist Josh Billings“To know that you do not know is the best. To pretend to know when you do not know is a disease.” Lao-Tzu"It ain't what you know that hurts you. It's what you do know that ain't so." Will RogersI'm sure Mark Twain, Winston Churchill, and Disraeli had a version of the same quote.
-- Fred Van Bennekom, Dr.B.A., Principal Great Brook Consulting
Please contact us with feedback on this article or other topics that you might find interesting.
Click here to add your name to our Service Insights newsletter which highlights newly published articles.