Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University
This question gets at an important consideration in the planning of a SoTL project. What is my source of evidence? Will I use data from focus groups, surveys, student reflections, or something else? Will this evidence focus on student self-reports/perceptions of learning or will the evidence be more objective? The best guidance is that your evidence should match the purpose of your SoTL study. If you are seeking to understand students’ perspectives on a learning experience, then the evidence you collect should align with this. If, however, you are seeking to measure student learning, other forms of data may be more advantageous.
When SoTL-ists talk about their data, they can generally ascribe one of two labels to their evidence: direct or indirect. Direct evidence comes from objective sources such as classroom artifacts (e.g., exams/quizzes, projects/assignments), systematic observations (e.g., video/in-person observations, photographs), or student reflections that tell the story of their own attitudes or beliefs. Indirect evidence is sourced from more subjective sources – student reports of their own learning, teacher reflections of student learning (Vanderbilt, 2013). So, to return to the excellent question posed to me above, teacher/learner perceptions CAN be a reliable data source if the SoTL work in question seeks to understand how teachers/learners feel about their learning. That said, if a researcher is seeking to identify changes in student learning, perceptions alone are not a strong form of evidence to study (see this blog post from 2015 for an expanded discussion of this notion).
One of the best resources I’ve found to explain the difference in various evidence types in SoTL was published by Vanderbilt University’s Center for Teaching. This resource, Gathering Evidence: Making Student Learning Visible, outlines the difference between direct and indirect evidence clearly and cogently, providing examples and brief explanations to understand these concepts well. For my upcoming workshop, I adapted and converted the information shared on this resource (giving ample credit to Vanderbilt!) into a decision tree to share with the SoTL mentors I’ll be working with. As SoTL mentors, they will need to be well informed as to the pros and cons of direct and indirect evidence. I’m hopeful this visual will give us a good starting point for that discussion!
As a plug for upcoming blogs, additional information is coming in October and November on methods to consider evidence in new and different ways…stay tuned! I am certain that most of the methods that will be covered will apply predominantly to analysis of direct evidence in the study of teaching and learning.
Vanderbilt University Center for Teaching. (2013). Gathering evidence: Making student learning visible. Available at: https://my.vanderbilt.edu/sotl/files/2013/09/4SoTLEvidence.pdf