Hatch, J.A. (2002)
Creswell
http://www.sagepub.com/booksProdDesc.nav?prodId=Book235677
Determining Validity in Quality Inquiry
Creswell, J. W., Miller, D. L., Creswell, J. W., & Miller, D. L. (2010). in Qualitative Inquiry, (February 2014), 37–41. doi:10.1207/s15430421tip3903
Reading notes:
The term and definition of validity was elaborated. For qualitative research at least it is agreed that studies should be “credible” with which common procedures were identified, checking, triangulation, thick description, peer reviews and external audits etc. However the selection among these procedures against each other is not well differentiated. In this paper, two perspectives are suggested to guide the choice of procedures.
A two-dimensional framework as a rationale for choice of a procedure based on
- who assessses the credibility of a study, and (so it’s about standard & requirement to fulfill?)
- one’s own philosophical positions towary qualitative inquiry (so this is an approach with self perceived, reflected and justified rationale and reasons for the research and its methods? It gives space for an individual to discuss and share about the self-constructed realities.)
Thus the author assumed that validity refers not to the data but the inferences they drawn (Hammersley & Atkinson, 1983).
A comparison made regarding lens used by researchers
- Quantitative: inferences made from test scores on psychometric instruments (construct, criterion, and content validity of interpretation of scores) (AERA, APA, & NCME, 1982)
- Qualitative: a lens established using the views of people who conduct, participate in, or read and review a study. It is a “validity-as-reflexive-accounting” approach as Altheide and Johnson (1994) described as where researchers, the topic and the sense-making process interact.
Overview of three paradigm assumptions.
Postpostivist – rigorous methods, systematic forms of inquiry; recognize and support validity, look for quantitative equivalence of it, and actively employ procedures for establishing validity using specific protocols.
Constructivist or interpretive – believe in pluralistic, interpretive, open-ended, and contextualized (e.g., sensitive to place and situation) perspectives toward reality. Criteria or qualities embraced such as as trustworthiness (i.e., credibility, transferability, dependability, and confirmability), and authenticity (i.e., fairness, enlarges personal constructions, leads to improved understanding of constructions of others, stimulates action, and empowers action).
Critical perspective – researchers should uncover the hidden assumptions about how narrative accounts are constructed, read, and interpreted. What governs our perspective about narratives is our historical situatedness of inquiry, a situatedness based on social, political, cultural, economic, ethnic, and gender antecedents of the studied situations. The implication for validity of this perspective is that validity is called into question, its assumptions interrogated and challenged, and the researchers need to be reflexive and disclose what they bring to a narrative.
9 types of validity procedures were tabled and mapped with respective paradigms taking the lens of researcher, study participants and people external to the study.
- Triangulation
Member checking
The audit trail - Disconfirming Researcher
Prolonged engagement in the field
Thick, rich description - Researcher reflexivity
Collaboration
Peer debriefing
So finally it is about how we position ourselves with our research study. We may consider the ease of use but also the target audiences, availability and access of the resources and sources, and expenses of using the procedures. More importantly as the authors mentioned we as inquirers should become more reflexive in our studies, understanding the existence and relationship of researcher and the process of inquiry etc.
Thus the search for validity should work beyond the use of validity procedures but the meta-cognitive knowledge of oneself, his/her roles in the research and relationship with the ones involved in the research, and the choice of procedures and the beliefs behind.
Insider research: Good or Bad?
The topic of challenges or double-edged sword of insider research is very interesting. It is an awareness and critical reflection of the positions, roles & duties of a researcher which could be regarded as conflicting and/or confounding in some areas.
Research I think originated from a curiosity in knowing more and better the world where we live with things and happenings in it and we ourselves as the being. In a very board sense, we research things that all relate to us with which we would have a specific position, role and duty anyway. It is hard to have a research irrelevant to us. The distance by all means, short or big, is comparative. The insiderness depends on specific emphasis and focus on certain attributes. Whether it is a strength or weakness depends very much on the criteria of good research, the actual implementation of the research and how explicit and clear it makes in its presentation or narrative.
Thus the two papers prompt me to revisit the criteria of good research.
(1) Researcher’s reflexivity in identifying any bias, preconception, single perspective which could bring forth subjectivity
(2) The access to the research subjects and data may create opportunity for easy access but it may not contribute to generalizability or generalization
(3) Ethics – whether or not the research is properly done without infringing any stakeholders or constituting any violation of personal or data privacy, abuse of power and authority etc.
All these areas would affect the validity and reliability of the research with which the implementation is crucial and needs to be made more explicit if not transparent.
By nature, research itself is good and should aim good. Thus this is regardless of the insiderness. As we should encourage everyone to embrace the idea and practice of research. It is also a professional reflective practice for learning and continuous development.
Fox’s Criteria of Good Research regarded research as one of the means to discover the truth and he dated three basic approaches to accomplish that. They are authority, logic, and controlled observation. He also argued that there were a lot of limitations and so research rarely reveals the whole truth. He suggested to view research findings as a means of approximating the truth instead. In this sense, I think it is no harm to have insider or outsider doing research as research itself is a mean to get closer to the truth. He had concerns on quality of the research as well. “Since research efforts vary widely in quality, the question of how much confidence can be placed justifiably in the findings of a particular research is one of the considerable importance.”
Thus he listed out 7 criteria which in his eyes as something of common-sense to differentiate “research that merits a good deal of confidence” from those with findings acceptable with reservations.
(1) A clearly defined and stated research purpose or problem (the least ambiguity)
(2) Research procedures be described in sufficient details for repeating the research
(3) A carefully planned procedural design of the research for data and results yielded as objective as possible
(4) Report with complete frankness from the researcher on any flaws in the procedural design and estimation on their effects on findings
(5) Data analysis should be adequate to reveal its significance; and methods of analysis should be appropriate
(6) Conclusions be confined to those justified by the data of the research and limited to those for which the data provides an adequate basis
(7) Greater confidence in the research is warranted if the researcher is experienced, has a good reputation in research, and is a person of integrity
Brannick, T. and Coghlan, D. (2007). In Defense of Being “Native”: The Case for Insider Academic Research. Organizational Research Methods, 10 (1), 59-74.
Fox, J. H. (1958). Criteria of good research. The Phi Delta Kappan, 39(6), 284-286.
Mercer, J. (2007). The challenges of insider research in educational institutions: wielding a double-edged sword and resolving delicate dilemmas. Oxford Review of Education, 33 (1), 1-17.
Robert Gagne’s Nine Events of Instruction
The Center for Instructional Technology & Training applies Gagne’s Nine Events of Instruction to regular, large and online & hybrid courses. A lot of practical tips for instruction.
Notes: 4C/ID Model ( van Merrienboer)
Four complement instructional design (4C/ID)
For complex learning (Merrienboer & Kirschner, 2012)
Components – steps to complex learning
– learning tasks
– supportive info
– procedural info
– part-task practice
….
Notes: Cognitive apprenticeship – Collins et. al
Basic notion
– learning in a specific context requires making implicit thinking and learning processes VISIBLE, and providing GUIDANCE & assistance to learner when necessary
– early learners are provided a great deal of learning support
– more advanced learners are provided a great deal of freedom to explorer and devise their own solutions
?
Roles & relationship
Development
Masters/Expert
Novice
Situated learning
Modeling
Scaffold
Six methods
– modeling
– coaching: FEEDBACK
– scaffolding: a designed & structured context for learning, ZPD
– articulation
– reflection:
– exploration: new application
Notes: Nine Events of Instruction (Gagne)
Instructional events to learning process
– attention
– objective
– prior learning
– present of content
– learning guidance
– elicit performance ?
– feedback
– assess performance
– enhance retention & transfer (generalization of … skills, knowledge, strategies/approaches to problem solving)
To read: Merrill’s instruction theory
Why Basic Principles of Instruction Must Be Present in the Learning Landscape, Whatever
Form It Takes, for Learning to Be Effective, Efficient and Engaging (M. David Merrill)
http://blogs.sch.gr/stelam/files/2011/06/MERRILL-PRINCIPLES-OF-INSTRUCTION.pdf
Notes: Theories of instruction
Five principles of instruction – Merrill
– learners should be engaged in solving meaningful problems
– existing learner knowledge pertinent to the task should be activated
– new knowledge and skills should be demonstrated in an appropriate context
– learners should have ample opportunity to apply new knowledge and skills with feedback
– (application & transference) learners should be encouraged to make the knowledge and skill a part of their everyday world whenever possible
Dear Amia EdD 