This blog serves as the beginning of a four-week focus on unique research methods for SoTL work. Enjoy, and please feel free to write to our guest bloggers with any feedback or questions! -Jen Friberg, blog editor
I recently completed a SoTL project where I was hoping to create vicarious, but meaningful, applications of classroom learning, in this case, foundational theories of the human development field. In an attempt to accomplish this, I utilized interrupted case studies (ICS), a progressive disclosure of information viewed as problem-based learning over time. Over an eight week period following a pre-test application, students viewed a longitudinal series of interviews as an ICS. This series followed several participants from the time they were seven years old in 1964, revisiting them every seven years until age 56 in 2013. During the process, and using the assumptions, concepts, and language of assigned developmental theorists, students described and applied relevant theoretical positions to anticipate growth and change as this collection of real lives progressed. Their work was submitted in weekly reflective essays. At the end of the eight-week assignment, post-test results indicated that the method was quite successful but told me nothing further. The post-test increase could simply be the result of memorizing the material. Therefore, pattern-matching was applied to further examine those results.
Pattern-matching is a less-known, but dependable, procedure for theory testing with case-studies and is regularly recommended for reconciling mixed methods and data sources in case study research, and to boost the rigor of the study. The overarching goal is to explain relationships between key points, in this case the pre/post results, by comparing an identified theoretical pattern with an observed pattern.
The previously mentioned weekly student essays were utilized as the observed pattern. These included descriptions of their assigned interview participants, appraisals of their most recent developmental predictions for this person, and their expectations for the next seven years. The essays were coded line-by-line to determine the degree of matching to a predetermined theoretical pattern. In this case, Bloom’s Revised Taxonomy (Anderson & Krathwohl, 2001) was utilized as coding categories as follows: 1 – Remembering, 2 – Understanding, 3 – Applying, 4 – Analyzing, 5 – Evaluating, and 6 – Creating. Average use of these levels could show a general progression from simple remembering (e.g. defining, telling, listing) to application (identifying, selecting, organizing) to creating (imagining, elaborating, solving). Once the essays were coded, interrater reliability was determined by using the intra-class correlation coefficient function of SPSS v. 20 to determine a kappa score of reliability, with a score of .80 deemed reliable.
Results, the observed pattern, allowed me to see a progress toward more complex reasoning in the assignment as the class progressed and students gathered more information and became more comfortable with theory application. Briefly stated, the first essays indicated an average response at Bloom’s applying level. Students were identifying and correctly applying concrete elements of the theories and making tentative, but informed, inferences. However, by the final essay the average response level was solidly at the evaluating level. There, students were appraising the flexibility of the theories being applied along with the documentary participant they were following. It became more common to see students suggest multiple possibilities in their writing, prioritize these, and determine the most informed interpretation. Consequently, pattern-matching indicated an established theoretical progression in reflective thinking from pre- to post-test.
Still, very few specific examples of best-practice exists with pattern-matching (Almutairi et al, 2014) and applications in SoTL (and education, in general) are rare. However, there are a number of available theories that could be considered as an identified pattern. For instance, I am currently using William Perry’s (1999) scheme of intellectual develop during the college years as a pattern basis in order to better understand contemporary student’s willingness, or unwillingness, to discuss racism in the classroom. Perry’s scheme is noticeably related to Bloom’s work, though somewhat better suited to assess student readiness to learn. Lastly, there are several other established variations of pattern-matching. In you are interested, a good place to begin would be Robert K. Yin’s (2009), Case study research: Design and methods.
Almutairi, A.F., Gardner, G.E., & McCarthy, A. (2014). Practical guidance for the use of pattern-matching technique in case-study research: A case presentation. Nursing and Health Sciences, 16, 239-244. doi: 10.1111/nhs.12096.
Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. London: Longman, Inc.
Perry, W. (1999). Forms of ethical and intellectual development in the college years: A scheme. San Francisco: Jossey-Bass.
Yin, R.K. (1994). Case study research: Design and methods (2nd Ed.). London: Sage.