The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…

Leave a comment

Direct vs. Indirect Evidence of Student Learning

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

measure2Later this week, I have the opportunity to facilitate a workshop on peer mentorship in SoTL at a nearby university. I solicited questions from mentor faculty as part of my workshop planning process. In doing so, one of the most interesting questions I received was the following: In studying student learning, how can teacher/learner perceptions be considered a reliable data source?

This question gets at an important consideration in the planning of a SoTL project. What is my source of evidence? Will I use data from focus groups, surveys, student reflections, or something else? Will this evidence focus on student self-reports/perceptions of learning or will the evidence be more objective? The best guidance is that your evidence should match the purpose of your SoTL study. If you are seeking to understand students’ perspectives on a learning experience, then the evidence you collect should align with this. If, however, you are seeking to measure student learning, other forms of data may be more advantageous.

When SoTL-ists talk about their data, they can generally ascribe one of two labels to their evidence: direct or indirect. Direct evidence comes from objective sources such as classroom artifacts (e.g., exams/quizzes, projects/assignments), systematic observations (e.g., video/in-person observations, photographs), or student reflections that tell the story of their own attitudes or beliefs. Indirect evidence is sourced from more subjective sources – student reports of their own learning, teacher reflections of student learning (Vanderbilt, 2013). So, to return to the excellent question posed to me above, teacher/learner perceptions CAN be a reliable data source if the SoTL work in question seeks to understand how teachers/learners feel about their learning. That said, if a researcher is seeking to identify changes in student learning, perceptions alone are not a strong form of evidence to study (see this blog post from 2015 for an expanded discussion of this notion).

One of the best resources I’ve found to explain the difference in various evidence types in SoTL was published by Vanderbilt University’s Center for Teaching. This resource, Gathering Evidence: Making Student Learning Visible, outlines the difference between direct and indirect evidence clearly and cogently, providing examples and brief explanations to understand these concepts well. For my upcoming workshop, I adapted and converted the information shared on this resource (giving ample credit to Vanderbilt!) into a decision tree to share with the SoTL mentors I’ll be working with. As SoTL mentors, they will need to be well informed as to the pros and cons of direct and indirect evidence. I’m hopeful this visual will give us a good starting point for that discussion!

Direct vs indirect decision tree

As a plug for upcoming blogs, additional information is coming in October and November on methods to consider evidence in new and different ways…stay tuned! I am certain that most of the methods that will be covered will apply predominantly to analysis of direct evidence in the study of teaching and learning.

Blog Reference

Vanderbilt University Center for Teaching. (2013). Gathering evidence: Making student learning visible. Available at:





Leave a comment

SoTL Book Information: ISU Faculty Authors

The announcement below is a press release from the Mathematical Association of America (MAA) that has just published a book on SoTL in Mathematics in higher education. This book could serve as a model for such books in other disciplines. In addition, chapter 17 was written by ISU faculty members in our department of Mathematics: Chapter 17. Mathematics Research Experiences for Preservice Teachers: Investigating the Impact on Their Beliefs by Wendy A. O¹Hanlon, David D. Barker, Cynthia W. Langrall, John A.Dossey, Sharon M. McCrone, and Saad I. El-Zanati. Another book focusing on SoTL in a particular discipline is also a model and one author is also an ISU faculty member (Jennifer Friberg): Ginsberg, S., Friberg, J., & Visconti, C. 2012. Scholarship of Teaching and learning in speech-language and audiology: Evidence-based education. San Diego: Plural Publishing. It is great to see ISU faculty take leadership roles in promoting SoTL in their disciplines.

A new book from the Mathematical Association of America (MAA) serves as a how-to guide for collegiate mathematics faculty who want to know more about conducting scholarly investigations into their teaching and their students’ learning. Out this month as part of the MAA’s Notes series, Doing the Scholarship of Teaching and Learning in Mathematics aims to both assist mathematics faculty interested in undertaking scholarly study of their teaching practice and promote a greater understanding of this work and its value to the mathematics community. The volume was envisioned and edited by Jacqueline Dewar and Curtis Bennett (Loyola Marymount University).

The Scholarship of Teaching and Learning (SoTL) movement encourages faculty to view difficulties encountered in the classroom as invitations to conduct research. In this growing field of inquiry, faculty bring their disciplinary knowledge and teaching experience to bear on questions of teaching and learning. They systematically gather evidence to develop and support their conclusions. The results are peer reviewed and made public.
The four chapters in Part I of Doing the Scholarship of Teaching and Learning in Mathematics provide background on this form of scholarship and specific instructions for undertaking a SoTL investigation in mathematics. Part II contains 15 examples of SoTL projects in mathematics from 14 different institutions, both public and private, spanning the spectrum of higher educational institutions from community colleges to research universities. The final chapter offers the editors’ synthesis of the contributing authors’ perceptions of the value of SoTL.

“Dewar and Bennett’s volume gives a vivid overview of the fresh field of the Scholarship of Teaching and Learning,” says Frank Farris (Santa Clara University). “It provides exactly what you need to get started doing research with your own classroom as the laboratory.”
Dewar, J., & Bennett, C. (Eds.). 2015. Doing the scholarship of teaching and learning in mathematics. Washington, DC: Mathematical Association of America.)