The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Interrupted Case Studies and Unexpected Results

Written by Bill Anderson, Associate Professor in the Department of Family and Consumer Sciences at Illinois State University

Note: Work on this project was supported by a 2016 SoTL Small Grant from the Office of the Cross Endowed Chair in SoTL at ISU

It is generally agreed that the value of fundamental course content is as a means to deeper learning. Specifically, learning to successfully apply the content to problem-solving and to transfer knowledge to future, more meaningful applications. Often, the most significant improvement teachers can make is giving students more application opportunities and real-world observational experiences related to the subject material. However, less class time is typically available for this perhaps more meaningful learning. When authentic application is impractical or not possible, other forms of classroom doing and observing can be valuable. Past experience has taught me that utilizing case studies can help provide an opportunity to apply content that students might not have otherwise.

From this idea, I began searching for available case studies relevant to one of my graduate classes, FCS 408 – Human Development in Social Context. Finding very few satisfactory materials, I decided to use a long-lived documentary, 56-Up, directed by Michael Apted, as an interrupted video case study (IVCS). 56-Up follows several children from the time they were seven-year-olds in 1964, revisiting them every seven years until age 56, in 2013. Each week students encountered the same group of real people, seven years older. Following this interrupted case-study format, and using the subject matter of the course, students applied relevant theoretical positions to anticipate growth and change as a collection of unique lives progressed. Students worked with incomplete information, made preliminary predictions, gathered missing information, refined their hypotheses, and continued with tentative predictions in weekly reflective essays and class discussions.

A pre-test consisting of 39 questions relating to established theories in the Human Development field (i.e. Erikson’s Psychosocial Development, Perry’s Reflective Judgment, Bronfenbrenner’s Bioecoogical model) was administered prior to viewing 56-Up. Post-test results at the completion of the documentary indicated essentially a two letter grade improvement from pre-test. These results were further substantiated utilizing a Pattern-Matching analysis (Yin, 2009) that indicated a significant increase of critical thinking across the seven-week period. However, there were unanticipated findings as well, as the IVCS was found to successfully address limitations frequently associated with case-based instruction (CBI). For instance, Mayo (2004) cited two common limitations with CBI addressed in this study:

  • most case studies are limited in length and relate to only a few course concepts
  • because many case studies are fictional, students may likely find it less valuable.

56-Up reflects real lives over almost 50 years, making it possible to apply multiple important theories across the lifespan. Results also indicated that narratives of actual events are indeed perceived by students as more engaging and evocative.

Egleston (2013) also sees CBI as somewhat limited, reporting that if a student is presented with a case study as a set of questions, what is very likely being assessed is simply the student’s ability to locate predetermined answers openly available within the case itself. In such an instance, Egleston reports that students do not learn where, or how, to ask appropriate questions; they learn to answer those asked by others. In effect, they simply learn that the answers are in front of them. In addition, it appears that many currently available case studies now have student responses, instructor write-ups, and class presentations readily available online. So, instructors must be aware of this possibility. 56-UP, as an IVCS, was wholly developed in real time with no readily available answers for students to cite or easily download.

Finally, because I served in the role of both teacher and researcher for this particular experiment, and despite my efforts to control factors that could have influenced the results, I acknowledge the potential for experimenter bias. Still, I am certainly pleased with the outcome and will continue using the interrupted format for the foreseeable future. I’ve also recently come across a very similar documentary, Angus Gibson’s 21-Up South Africa, which also shows great potential in teaching. I hope to begin developing this idea and use this second documentary next year.

 

Blog References

Egleston, D.O. (2013). The interactive, progressive case study. Business Education Innovative Journal, 5(1), 101-104.

Mayo, J.A. (2004). Using case-based instruction to bridge the gap between theory and practice in psychology of adjustment. Journal of Constructivist Psychology, 17, 137-146.

Yin, R.K. (2009). Case study research: Design and methods. London: Sage.


Leave a comment

Application of SoTL: Sharing Results with Students

Written by Susan Hildebrandt, Associate Professor of Applied Linguistics/Spanish, Department of Languages, Literatures, & Cultures at Illinois State University 

“Understanding World Language edTPA,” a two-hour workshop I presented at the annual meeting of the Illinois Council on the Teaching of Foreign Languages (ICTFL) in Tinley Park, focused on the content-specific student teacher performance assessment purported to measure beginning teacher readiness. edTPA became consequential for every individual seeking teacher licensure in the state of Illinois in September 2015. Student learning was central to this workshop as it explored how ISU world language teacher candidates performed on edTPA. This systematic study of ISU student learning is timely for world language teacher education programs throughout the state. By examining and sharing my students’ performance on the standardized edTPA, a state-wide audience learned from their triumphs and challenges. The workshop also served as an opportunity for a variety of audiences to get a wider view of edTPA, its origin, and its use.

The intended audience for this presentation was world language teacher education coordinators or world language pedagogy instructors and faculty, but a much more diverse audience attended the session. Five of the nine attendees were world language teacher candidates from across the state, who were taking pedagogy classes this semester and intended to student teach during the spring of 2107. The purpose of this workshop was originally to help world language teacher education programs get their students ready for edTPA. Instead, I got to go straight to the intended audience. The edTPA outcomes of my students were able to communicated to teacher candidates directly and I was able offer practical suggestions about how be more successful at demonstrating effective K-12 teaching practices. I was able to point out the areas in which my candidates were successful and those in which they struggled. I was able to share resources that were of particular value to my teacher candidates here at ISU.

The workshop deconstructed edTPA with an exploratory quantitative study, in which I examined edTPA scores of world language teacher candidates (N = 34) and compared their scores to the known cut scores for states in which edTPA is a requirement for licensure. Results indicated that participants performed best in the planning section of the assessment and were most challenged by the assessment section. All participants earned scores above the current minimal cut score for Washington state, and all but two would pass in New York state. The workshop also highlighted ways of encountering the three required tasks, along with logistical guidance for videotaping and writing the extensive commentaries for each task.

The teacher candidate attendees expressed great interest in these results, as more than one intended on teaching in another state. As a result of their interest, I decided to bring my findings back to my own class. I had intended to talk with them about edTPA that day, but I hadn’t intended to give them a research presentation. And yet, I did. And I think they enjoyed it. It’s not often they get to peak behind the curtain of a teacher education program and see how we use data to improve practice. I hope, too, that my teacher candidates can use this experience to learn to analyze their own classroom-based data, one of the skills assessed in edTPA.


Leave a comment

Institutionalization of SoTL: Thinking About Outcomes at Two ISUs

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

Recently, Marcketti and Freeman (2016) published an article in the Journal of the Scholarship of Teaching and Learning focused on outcomes following adoption of promotion and tenure policies that support the scholarship of teaching and learning (SoTL) at Iowa State University. In their work, Marcketti and Freeman provide a wonderful summary of the issues impacting the institutionalization of SoTL, highlighting the need to develop consistent and visible reward structures for faculty engaging in SoTL. Specifically, these authors focus on the creation of SoTL-friendly promotion and tenure policies to acknowledge the value of SoTL work.

Marcketti and Freeman (2016) provide the language from Iowa State University’s Faculty Handbook, which offers the following guidance:

  • SoTL is valued and should be held to similar standards of rigor and peer review as other, disciplinary research and/or creative activity.
  • While all faculty should engage in scholarly teaching, not all faculty need engage in SoTL.
  • If a faculty member does choose to pursue a research agenda that includes SoTL, all SoTL work “counts” as scholarship and/or creative activity, rather than as a part of assigned teaching responsibilities.

This language serves to promote, extend, and support SoTL at Iowa State. This is evidenced by the fact that five-year averages calculated by Marcketti and Freeman (2016) for faculty engagement in SoTL have ranged from 44-52% for faculty on their campus (see article for variation by faculty seniority and type of SoTL work). I view these data as remarkable and think that those involved in the process of developing this supportive and productive environment for SoTL at Iowa State should be commended.

Thinking about this work several days after my initial read of Marcketti and Freeman’s article, I found myself wondering how typical these outcomes are at other institutions in terms of faculty involvement and engagement in SoTL. Closer to home, I considered the current SoTL support structures at my own university and have pondered what else I might do to proactively support SoTL at Illinois State University.

Harkening back to my days as a school-based speech-language pathologist, I often worked with children to help them reflect on their learning using a “KWL” chart. In doing so, I encouraged students to identify what they knew (K), what they wanted to know (W), and (after an experience) what they learned (L). In reading Marcketti and Freeman’s work, I considered the work done at Iowa State from an adapted KWL perspective to perhaps illuminate future efforts at Illinois State and other institutions:

  1. What mechanisms do you have in place to support SoTL at your institution?
  2. What processes can be developed to establish and extend support for SoTL on your campus? How can these be developed?
  3. What are the outcomes of these supports? Have they served to increase SoTL engagement and support? In what ways?

I think that attention to this last item — outcomes of the supports in place for SoTL — is critical. One basic rationale for SoTL is that we can’t assume that learning happens just because we think it does in our classrooms. Similarly, I would argue that we can’t assume that faculty engagement happens simply due to the provision of support for SoTL. Rather, we need to evaluate the mechanisms that are put into place to identify those most successful at our individual institutions.

Faculty engaged in SoTL at Illinois State University have access to research grants, travel grants, workshops/trainings, consultation, publishing opportunities, social media support, and a robust SoTL-specific website. Our institutional strategic plan, Educating Illinois, specifically mentions the need to grow SoTL on campus. With these numerous supportive mechanisms in place, I am unsure which are most helpful for faculty, individually or collectively. There is work to be done to examine outcomes from these programs and initiatives. Additionally, while I am aware that many departments/schools at Illinois State University support and value SoTL, I am not certain whether any specifically mention SoTL as part of their promotion and tenure policies and procedures. Thus, there is additional work to be done to understand the impact of supportive reward structures at my institution.

Thanks to Marcketti and Freeman for their article and their work at Iowa State University. I appreciate the fact that colleagues from the “other” ISU helped me to think about efforts to support SoTL on my campus from a new and different perspective.

Blog References:

Marcketti, S. B. & Freeman, S. (2016). SoTL evidence of promotion and tenure vitas at a             research university. Journal of the Scholarship of Teaching and Learning, 16(5), 19-31.


Leave a comment

Facilitating Metacognitive Awareness Via Perspective Taking

Written by Lisa Vinney, Assistant Professor of Communication Sciences and Disorders at Illinois State University

It’s no secret that metacognition, or thinking about thinking to critically evaluate and shape one’s future learning and behavior, is tied to strong learning gains and greater success on a range of cognitive tasks (Dunlosky & Metcalfe, 2009; Koriat, Ackerman, Locke, & Schneider, 2009; Prins, Veenman, & Elshout, 2006; Nelson & Dunlosky, 1991; Koriat & Bjork, 2006; Koriat, 2008). Thus, developing and implementing innovative pedagogies to support students’ metacognitive awareness is a worthwhile endeavor. Borne out of this idea, Jennifer Friberg, current Cross Endowed Chair for SoTL at Illinois State University, and I designed and implemented an independent study for undergraduate Communication Sciences and Disorders students during Spring 2016. This experience was meant to promote deep reflection and metacognitive awareness in relationship to the devastating effects of laryngeal cancer and the complexities of its interprofessional management. Support for studying the results of the pedagogical methods included in this independent study (i.e. quantitative analysis of our results, qualitative coding, and preparation of a presentation for ISSoTL) was provided by a SoTL Research Mini-Award by the immediate past Cross Endowed Chair, Kathleen McKinney.

Accordingly, our work involved offering a semester-long experience to seven undergraduate students who engaged in weekly discussions related to topical, assigned readings on various aspects of laryngeal cancer. During discussions, students engaged with case scenarios and then were assigned the role of one or more individuals from the case (e.g., patient, family, and other professionals). perspectiveThey then answered questions and engaged in discussion about aspects of each case from the perspective of their assigned role. Following each weekly reading and discussion, students answered five standard reflection questions. These reflections were used as a vehicle to determine the efficacy of our pedagogical methods along with a questionnaire administered as a pre- and post- measure called the Metacognition Awareness Inventory (MAI; Schraw and Dennison, 1994). Specifically, we qualitatively analyzed changes in weekly student reflections across the independent study and quantitatively analyzed changes in MAI scores pre and post independent study.

Our pilot data, recently presented at ISSoTL in Los Angeles, indicated that the content of student reflections evolved from a focus on foundational metacognitive knowledge (i.e. student’s knowledge about useful learning strategies and their own personal learning style) to self-regulatory components of metacognitive awareness (i.e. knowledge about planning, monitoring, evaluating, and engaging in specific strategies to facilitate and manage learning and cognition). Currently, we are implementing the same independent study experience with a new group of 10 students. After the completion of the semester, we will again analyze students’ pre- and post-MAIs and weekly written reflections to add to our pilot data. That said, our preliminary results alone indicate that opportunities for discussion-based perspective taking followed by independent written reflections may promote discipline-specific content knowledge as well as higher-level metacognitive processes.

Blog References

Dunlosky, J. & Metcalfe, J (2009). Metacognition. Los Angeles, CA: Sage Publishing.

Koriat, A. (2008). Easy comes, easy goes? the link between learning and remembering and its exploitation in metacognition. Memory & Cognition, 36(2), 416-428.

Koriat, A., & Bjork, R. A. (2006). Mending metacognitive illusions: A comparison of mnemonic-based and theory-based procedures. Journal of Experimental Psychology. Learning, Memory, and Cognition, 32(5), 1133-1145.

Koriat, A., Ackerman, R., Lockl, K., & Schneider, W. (2009). The memorizing effort heuristic in judgments of learning: A developmental perspective. Journal of Experimental Child Psychology, 102(3), 265-279.

Nelson, T. O., & Dunlosky, J. (1991). When people’s judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The ‘delayed-JOL effect.’ Psychological Science, 2(4), 267-270.

Prins, F. J., Veenman, M. V. J., & Elshout, J. J. (2006). The impact of intellectual ability and metacognition on learning: New support for the threshold of problematicity theory. Learning and Instruction, 16(4), 374-387.

Schraw, G. & Dennison, R.S. (1994). Assessing metacognitive awareness. Contemporary Educational Psychology, 19, 460-475.