The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Theoretical Pattern-Matching in SoTL: SoTL Methods Series #1

This blog serves as the beginning of a four-week focus on unique research methods for SoTL work. Enjoy, and please feel free to write to our guest bloggers with any feedback or questions! -Jen Friberg, blog editor

Written by: Bill Anderson, Associate Professor of Family and Consumer Sciences at Illinois State University (jander2@ilstu.edu)

I recently completed a SoTL project where I was hoping to create vicarious, but meaningful, applications of classroom learning, in this case, foundational theories of the human development field. In an attempt to accomplish this, I utilized interrupted case studies (ICS), a progressive disclosure of information viewed as problem-based learning over time. Over an eight week period following a pre-test application, students viewed a longitudinal series of interviews as an ICS. This series followed several participants from the time they were seven years old in 1964, revisiting them every seven years until age 56 in 2013. During the process, and using the assumptions, concepts, and language of assigned developmental theorists, students described and applied relevant theoretical positions to anticipate growth and change as this collection of real lives progressed. Their work was submitted in weekly reflective essays. At the end of the eight-week assignment, post-test results indicated that the method was quite successful but told me nothing further. The post-test increase could simply be the result of memorizing the material. Therefore, pattern-matching was applied to further examine those results.

patternPattern-matching is a less-known, but dependable, procedure for theory testing with case-studies and is regularly recommended for reconciling mixed methods and data sources in case study research, and to boost the rigor of the study. The overarching goal is to explain relationships between key points, in this case the pre/post results, by comparing an identified theoretical pattern with an observed pattern.

The previously mentioned weekly student essays were utilized as the observed pattern. These included descriptions of their assigned interview participants, appraisals of their most recent developmental predictions for this person, and their expectations for the next seven years. The essays were coded line-by-line to determine the degree of matching to a predetermined theoretical pattern. In this case, Bloom’s Revised Taxonomy (Anderson & Krathwohl, 2001) was utilized as coding categories as follows: 1 – Remembering, 2 – Understanding, 3 – Applying, 4 – Analyzing, 5 – Evaluating, and 6 – Creating. Average use of these levels could show a general progression from simple remembering (e.g. defining, telling, listing) to application (identifying, selecting, organizing) to creating (imagining, elaborating, solving). Once the essays were coded, interrater reliability was determined by using the intra-class correlation coefficient function of SPSS v. 20 to determine a kappa score of reliability, with a score of .80 deemed reliable.

Results, the observed pattern, allowed me to see a progress toward more complex reasoning in the assignment as the class progressed and students gathered more information and became more comfortable with theory application. Briefly stated, the first essays indicated an average response at Bloom’s applying level. Students were identifying and correctly applying concrete elements of the theories and making tentative, but informed, inferences. However, by the final essay the average response level was solidly at the evaluating level. There, students were appraising the flexibility of the theories being applied along with the documentary participant they were following. It became more common to see students suggest multiple possibilities in their writing, prioritize these, and determine the most informed interpretation. Consequently, pattern-matching indicated an established theoretical progression in reflective thinking from pre- to post-test.

Still, very few specific examples of best-practice exists with pattern-matching (Almutairi et al, 2014) and applications in SoTL (and education, in general) are rare. However, there are a number of available theories that could be considered as an identified pattern. For instance, I am currently using William Perry’s (1999) scheme of intellectual develop during the college years as a pattern basis in order to better understand contemporary student’s willingness, or unwillingness, to discuss racism in the classroom. Perry’s scheme is noticeably related to Bloom’s work, though somewhat better suited to assess student readiness to learn. Lastly, there are several other established variations of pattern-matching. In you are interested, a good place to begin would be Robert K. Yin’s (2009), Case study research: Design and methods.

References

Almutairi, A.F., Gardner, G.E., & McCarthy, A. (2014). Practical guidance for the use of pattern-matching technique in case-study research: A case presentation. Nursing and Health Sciences, 16, 239-244. doi: 10.1111/nhs.12096.

Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. London: Longman, Inc.

Perry, W. (1999). Forms of ethical and intellectual development in the college years: A scheme. San Francisco: Jossey-Bass.

Yin, R.K. (1994). Case study research: Design and methods (2nd Ed.). London: Sage.

 

 

 

Advertisements


Leave a comment

Interrupted Case Studies and Unexpected Results

Written by Bill Anderson, Associate Professor in the Department of Family and Consumer Sciences at Illinois State University

Note: Work on this project was supported by a 2016 SoTL Small Grant from the Office of the Cross Endowed Chair in SoTL at ISU

It is generally agreed that the value of fundamental course content is as a means to deeper learning. Specifically, learning to successfully apply the content to problem-solving and to transfer knowledge to future, more meaningful applications. Often, the most significant improvement teachers can make is giving students more application opportunities and real-world observational experiences related to the subject material. However, less class time is typically available for this perhaps more meaningful learning. When authentic application is impractical or not possible, other forms of classroom doing and observing can be valuable. Past experience has taught me that utilizing case studies can help provide an opportunity to apply content that students might not have otherwise.

From this idea, I began searching for available case studies relevant to one of my graduate classes, FCS 408 – Human Development in Social Context. Finding very few satisfactory materials, I decided to use a long-lived documentary, 56-Up, directed by Michael Apted, as an interrupted video case study (IVCS). 56-Up follows several children from the time they were seven-year-olds in 1964, revisiting them every seven years until age 56, in 2013. Each week students encountered the same group of real people, seven years older. Following this interrupted case-study format, and using the subject matter of the course, students applied relevant theoretical positions to anticipate growth and change as a collection of unique lives progressed. Students worked with incomplete information, made preliminary predictions, gathered missing information, refined their hypotheses, and continued with tentative predictions in weekly reflective essays and class discussions.

A pre-test consisting of 39 questions relating to established theories in the Human Development field (i.e. Erikson’s Psychosocial Development, Perry’s Reflective Judgment, Bronfenbrenner’s Bioecoogical model) was administered prior to viewing 56-Up. Post-test results at the completion of the documentary indicated essentially a two letter grade improvement from pre-test. These results were further substantiated utilizing a Pattern-Matching analysis (Yin, 2009) that indicated a significant increase of critical thinking across the seven-week period. However, there were unanticipated findings as well, as the IVCS was found to successfully address limitations frequently associated with case-based instruction (CBI). For instance, Mayo (2004) cited two common limitations with CBI addressed in this study:

  • most case studies are limited in length and relate to only a few course concepts
  • because many case studies are fictional, students may likely find it less valuable.

56-Up reflects real lives over almost 50 years, making it possible to apply multiple important theories across the lifespan. Results also indicated that narratives of actual events are indeed perceived by students as more engaging and evocative.

Egleston (2013) also sees CBI as somewhat limited, reporting that if a student is presented with a case study as a set of questions, what is very likely being assessed is simply the student’s ability to locate predetermined answers openly available within the case itself. In such an instance, Egleston reports that students do not learn where, or how, to ask appropriate questions; they learn to answer those asked by others. In effect, they simply learn that the answers are in front of them. In addition, it appears that many currently available case studies now have student responses, instructor write-ups, and class presentations readily available online. So, instructors must be aware of this possibility. 56-UP, as an IVCS, was wholly developed in real time with no readily available answers for students to cite or easily download.

Finally, because I served in the role of both teacher and researcher for this particular experiment, and despite my efforts to control factors that could have influenced the results, I acknowledge the potential for experimenter bias. Still, I am certainly pleased with the outcome and will continue using the interrupted format for the foreseeable future. I’ve also recently come across a very similar documentary, Angus Gibson’s 21-Up South Africa, which also shows great potential in teaching. I hope to begin developing this idea and use this second documentary next year.

 

Blog References

Egleston, D.O. (2013). The interactive, progressive case study. Business Education Innovative Journal, 5(1), 101-104.

Mayo, J.A. (2004). Using case-based instruction to bridge the gap between theory and practice in psychology of adjustment. Journal of Constructivist Psychology, 17, 137-146.

Yin, R.K. (2009). Case study research: Design and methods. London: Sage.