Safeguarding sound design

Criteria for protecting the soundness of design

Ed 714 Qualitative Research Methods in Education
Spring/Summer 2001
Copyright - Antonia D'Onofrio - 2000/2001

The quality of qualitative studies is protected by consideration of the following issues:

decisions about breadth, depth and detail

the ability to account for rival explanations of data

the organizing framework

decisions regarding method and technique

specifics of method and technique

focus on human interaction

clarity of perspective

Specifics of method and technique

Are your decisions about breadth and depth consistent with the amount of information you plan to collect?
Have you provided for checks on reliability and accuracy?
Are you confident that your uses of informants are sound?
Have you considered how response biases may influence your findings? Can you interpret these biases?

Breadth, depth and detail
Will you study an entire problem or only part of it?
What interactions, processes will you explore?
Will your study be descriptive or exploratory?
What will be the level of concern for individuals versus groups?
What level of uncertainty is tolerable to you ?
Focus on human interactions
Have you accounted for your personal history with the problem?
Will you study the social dynamics between researcher and subjects?
Have the norms of interaction in the setting been explored as part of the study?
Have you reflected on how sub settings tie into realities in the larger group?
Will you conduct ongoing inferences that refine constructs intended to describe and explain human interactions?

Rival explanations - These are normally part of evaluations in the positivist paradigm. Each has been redefined in terms of its problem value for qualitative research.
mortality - How does systematic loss of subjects erode the quality of your information?
history - Do intervening events warp perceptions of setting, problems, individuals, researcher?
maturation - Will your respondents change over time in part because of their interaction with you, researcher, and the role of participant in a research study?
testing - Since you will study a group, person, setting over time, will repeated investigation change the quality of the study in any way?
measurement - How have biases related to you the observer been accounted for?
regression - Are you aware of how misleading your information may become if it is overly dependent on the typical, average or most frequent type of respondent or perspective?
selection - Forget about generalizability. Does your sampling technique use criteria that relate people events, time, and materials to the context of the study?
Clarity of perspective
Have you as researcher distinguished your own perspective on a problem from those of participants?
Will you analyze the differences?
Can you explain whether and why your research takes either an emic or etic perspective?
Are you confident that tacit understandings will be uncovered by your methods?
Do you have a plan for reformulating the nature of the problem at the end of the study?
Will your personal theory be revised?
Have you thought through a strategy for disconfirming assumptions made at the beginning of a study?
Will contradictions be explored and explained?

Organizing framework

Have you identified and defined important concepts and linked them?
Have you defined your problem in terms of bracketing assumptions that frame your personal understanding of the problem?
Have you described the history of a group and its people, including the impact of this history on the context of your problem?
Have you accounted for the defining attributes of a culture, group, organization?

The quality of qualitative studies is protected by consideration of the following issues: decisions about breadth, depth and detail the ability to account for rival explanations of data the organizing framework decisions regarding method and technique specifics of method and technique focus on human interaction clarity of perspective	Specifics of method and technique Are your decisions about breadth and depth consistent with the amount of information you plan to collect? Have you provided for checks on reliability and accuracy? Are you confident that your uses of informants are sound? Have you considered how response biases may influence your findings? Can you interpret these biases?
Breadth, depth and detail Will you study an entire problem or only part of it? What interactions, processes will you explore? Will your study be descriptive or exploratory? What will be the level of concern for individuals versus groups? What level of uncertainty is tolerable to you ?	Focus on human interactions Have you accounted for your personal history with the problem? Will you study the social dynamics between researcher and subjects? Have the norms of interaction in the setting been explored as part of the study? Have you reflected on how sub settings tie into realities in the larger group? Will you conduct ongoing inferences that refine constructs intended to describe and explain human interactions?
Rival explanations - These are normally part of evaluations in the positivist paradigm. Each has been redefined in terms of its problem value for qualitative research. mortality - How does systematic loss of subjects erode the quality of your information? history - Do intervening events warp perceptions of setting, problems, individuals, researcher? maturation - Will your respondents change over time in part because of their interaction with you, researcher, and the role of participant in a research study? testing - Since you will study a group, person, setting over time, will repeated investigation change the quality of the study in any way? measurement - How have biases related to you the observer been accounted for? regression - Are you aware of how misleading your information may become if it is overly dependent on the typical, average or most frequent type of respondent or perspective? selection - Forget about generalizability. Does your sampling technique use criteria that relate people events, time, and materials to the context of the study?	Clarity of perspective Have you as researcher distinguished your own perspective on a problem from those of participants? Will you analyze the differences? Can you explain whether and why your research takes either an emic or etic perspective? Are you confident that tacit understandings will be uncovered by your methods? Do you have a plan for reformulating the nature of the problem at the end of the study? Will your personal theory be revised? Have you thought through a strategy for disconfirming assumptions made at the beginning of a study? Will contradictions be explored and explained?
Organizing framework Have you identified and defined important concepts and linked them? Have you defined your problem in terms of bracketing assumptions that frame your personal understanding of the problem? Have you described the history of a group and its people, including the impact of this history on the context of your problem? Have you accounted for the defining attributes of a culture, group, organization?