The control of bias in qualitative studies

When does bias come into play? And when is it a problem?

During the problem identification stage

• The research question (s) need to be based on a realistic understanding of the problem as it is situated in a real context.

• our question does not place humans and human relationships at the core of the study.
• You have asked a self-serving question - one that reinforces stereotypes or current prejudices.
• You have asked a question that presumes its own conclusion as an answer.
• Your research question does not focus on aspects of human experience that lend themselves to qualitative methods.
• Your research question is not feasible.
• Your research question can be answered more effectively using nonqualitative methods.

• When formulating the conceptual framework or personal theory of the problem:

 
• A conceptual framework is a tool that allows a researcher to plan a study that exhibits internal coherence and leads in a logical way to emergent questions and methods.

• You don't allow for the problem of theoretical bias.
• Your conceptual model is not contextually sensitive to the realities of the research problem.
• You thoroughly invest in some else's theory instead of crafting your own conceptual model or modification of theory.
• You are not clear about the sources of knowledge that support your conceptual model
• You haven't discriminated among types of knowledge that you will use in crafting your conceptual model.
• You did not identify and define key concepts and issues that are the building blocks of your conceptual model.
• You did not discuss how key concepts and issues are conceptually connected.
• You forgot to explore, and make articulate, the assumptions that reside in your conceptual model.
• You develop primary and secondary research questions, but they don't emerge from your conceptual model.

 
• When selecting study participants:

 
• The setting or context may not convey a realistic framework for the problem and the choice of participants may be skewed.  The setting or context may not the best choice given the research problem.
• You may have chosen the wrong sampling method for the problem at hand.
• You used random sampling. (?)
• You may have underestimated the importance of sampling deviant members of a group.
• You may have underestimated the extent to which sampling the norm can skew your understanding.
• You may have forgotten that settings and events must also be sampled in a purposive way.
• You may have forgotten that texts, artifacts, and other things that people make and do must also be sampled.
• You underestimated the importance of using theoretical sampling to validate emergent concepts and patterns.

• When designing and developing information gathering strategies.
• Data collection needs to be systematic.  It also needs to be flexible and opportunistic.  The goal of data collection should be to collect information that will answer the research question (s), but it must also lead to authentic results.  Information should afford an internal perspective and be faithful to the perspectives and experiences of study participants.
• Your interview/observation strategies reflect your beliefs.
• You ignore the insights of individuals who have intimate knowledge of the social context of your research problem.
• You are too rigid.
• You are unsystematic.
• Your methods are not logically connected to your conceptual model/personal theory of the problem.
• You neglected to field test your methods.
• You forget to ask whether your chosen methods are contextually sensitive to your research problem.
• You fear unstructured situations and overly value closure.
• Your methods foreclose on the story that people in the field can tell.
• You plan follow up questions, interview probes, and follow up observations too far in advance
• You depend on only one method instead of multiple methods
• You depend on one type of evidence instead of multiple sources of evidence

• During active field work.
• Accuracy and consistency are important.  So is the task of overcoming one's assumptions and expectations.  It is important to gain access and establish rapport.  It is also important to analyze those processes.  Both researcher and methods have reactive effects on the setting.  These need to be investigated, as they are part of your knowledge base.  Bias is data.

• You may fail to take note of response biases
• You may be too structured in your approach and close off opportunities to explore important evidence.
• You may be unsystematic
• You may neglect to use member checks
• You may forget to build in reliability checks
• You may adopt an overly normative perspective
• You may have forgotten that you too were present in the research and had an effect on those around you.
• You may be too emeshed, forgetting that you are a researcher
• You forget that your biases are data and should be part of the evidence you investigate.
• You forget that the biases of study participants ar also data and should be respected as evidence.

• During record keeping.
• It is important to keep records that are detailed and comprehensive.  It is also important to have flexibility built in to record keeping, keeping in mind that a problem changes as one gets deeper into the problem itself.  Record keeping therefore should be friendly to new and unanticipated information.  Records also need to be accurate from the perspective of those whom we study.  They may have different views on what actually happened, was said or was heard.

• You forgot to develop a system of reliability checks
• You neglected to debrief you study participants
• You didn't think it was important to submit your notes to a dialogic test with individuals who have deep experience with the research problem in context.
• You were unsystematic
• Your record keeping wasn't flexible enough or open  to unexpected, but important, information.
• You relied on only one type of information (heard or written) forgetting that nonverbal information is also text.

• When analyzing information  Content analysis, for example.

• You were unable to connect categories or patterns
• Initial themes and patterns are demonstrated to be evident.  Emergent and patterns themes are absent.  Cultural meanings of words and terms are misunderstood or glossed over.  Cultural uses of language aren't identified and explored.
• When reporting findings

•  The report should recreate for readers the eye witness experiences of the researcher in the field.  The quality of realism is essential. A sense of place, perspective, person, emotion, relationship and coherence are essential ingredients to a believable result.

• No one's voice is heard.
• Only your voice is heard.
• Only your perspective is described.
• Few exemplars of interview data/observational data are included with descriptions of patterns and themes.
• The report offers only literal information about people and context.
• Themes, concepts and patterns that connect the evidence are missing.
• Themes and patterns are not linked to empirical evidence.
• Details are missing.
• Participant perspectives are not reported.
• Your presence in the study is not described.
• Connections between your thought processes, the ones you used to analyze the data, and categories, themes, and/or patterns haven't been described.

• During interpretation and theory building

• Research begins with assumptions, key ideas, principles and a fair amount of conceptual analysis regarding how assumptions and concepts are tied together.  Once concluded, a research report needs to show how questions were answered, old ideas have been challenged and new ideas and theories have been advanced.  A change in perspective should be obvious.  A researcher needs to discuss how his/her perspective has changed and how he/she has contributed to understanding.

• You were unable to connect categories or patterns
• Initial themes and patterns are discussed;  however, emergent and patterns themes are not.
• You haven't compared/contrasted your conclusions with arguments in the literature.
• Your interpretation affirms your expectations, yet surprising and contradictory trends go un-noticed.
• You didn't explore the unique qualities of individuals and groups.
• You didn't explore your personal change of perspective as researcher.
• You haven't identified new and emerging questions and problems.
• Your theory of the problem before and after the analysis of data are neither described nor explored.
• Assumptions coming from formal theory (if you used one) are not challenged, amended and/or expanded.
• A new theory of the problem is not advanced..

.return to the course schedule