Written by Erin Mikulec, SoTL Scholar-Mentor, Associate Professor (Teaching & Learning), and Interim Director of the English Language Institute at Illinois State University
Out-of-class learning offers many opportunities to examine student learning outcomes as the result of participation in various activities, ranging from student organizations, study abroad experiences, field-based or service learning projects associated with courses on campus. In order to support researchers who may be interested in engaging in this kind of SoTL work, this post outlines ways in which such a study might be designed. As a SoTL researcher, I have carried out several studies that have examined out-of-class learning and, in the process, have learned a great deal in regard to designing tasks and activities that serve as data measures. Although my own work has focused on study abroad and student organizations, the suggestions presented here may be applied to a variety of SoTL contexts for out-of-class learning.
As with any study, it is important to specify exactly what it is you want to know. When I first began studying out-of-class learning, my guiding questions were broad and general regarding student learning outcomes. However, in looking at out-of-class learning, one thing I’ve come to understand is the value of differentiating between personal and professional learning outcomes that students experience as a result of participation in such activities. For me as a researcher, this allowed me to examine student growth through different lenses beyond that of academics. Therefore, when thinking about your guiding questions for a study on out-of-class learning, think about how the project, experience or activity may impact students both professionally and personally. For instance, does the activity in the study develop students’ professional skills through a field-based experience? Which ones and how so? Or perhaps the experience supports students in their development of leadership and communication skills? Further still, does the experience allow for opportunities for personal growth and reflection? Although themes often emerge in data analysis, it can be helpful during the design process to be mindful of these differences in learning outcomes.
When thinking about how to design your out-of-class learning outcomes study, consider your initial data points. We know that good SoTL research considers multiple measures of data collection, thus, the means you use to capture this information at the on-set are important. These data measures can be used to identify where your students are in terms of skills and the potential for growth and development as the project or experiences progress. Some possible tasks and activities to do this include:
- Surveys, which can have Likert-scale items, open-ended questions, or both,
- Reflection exercises which help students to consider what they hope to gain from the project or experience, what they already know about a certain issue or topic, or what they are most excited about, apprehensive about, etc.,
- Online group discussions, based on materials provided by the researcher, such as videos, news or research articles about the topic of the project or experience, or simply questions posed by the researcher,
- Proposed work plans or schedules of tasks to be completed throughout the project or experience,
- Multimodal means of capturing students’ starting points, such as photos or videos that students create as part of their own documentation of the journey on which they are about to embark.
In my own out-of-class learning research, I have found that it is equally important to think of this as broadly as possible and more as a starting point rather than simply the “pre” in a pre-post scenario. In this way, using SoTL to assess out-of-class learning is more robust and not limited to a before and after picture of learning outcomes. This allows you as a researcher to examine a progression of learning outcomes and even, as will be discussed next, at which points in the project or experience these outcomes begin to take shape.
Once the project or experience has begun, consider the ways in which you can capture data as progress points along the way. The number of these data points and when you collect them will of course depend on the specifics of the project or experience and what you want to know in order to answer your guiding question(s). For instance, if it is a four week long project or experience, perhaps you are interested in tasks or activities that serve as data measures once a week or once every two weeks. Even a project or experience that is one day long can have data measures at different critical points throughout the time in which students are engaged. Regardless of how you design this aspect of your study, it will allow for more robust data collection and may reveal progress towards outcomes that support your findings in the end. Not only do in-progress measures serve as ways to collect more data, they can also point to issues arising within the project or experience that may require intervention, allowing the researcher even more insight into learning processes and outcomes as the result. These tasks and activities can be carried out individually or in pairs or groups. Some examples of means to do this are:
- Weekly reflections on how the project or experience is going, or what the students are experiencing,
- Diary or journal entries in which students discuss the highs and lows, the challenges and breakthroughs,
- Photos or videos,
- Observations of students engaged in various project or experience activities,
- Social media activity, for instance posts to Facebook or Twitter or blogs,
- Summaries of project tasks or activities in which the students have engaged.
As your project or experience draws to a close, what are some final sources of data that you can collect that will aid in answering your guiding questions? Consider carefully what these could be and what they could tell you about the out-of-class learning that has taken place over the course of your project or experience. These final tasks or activities could be summative in nature or more reflective. Perhaps they require that your students return to their initial thoughts and ideas about the project or experience and how those have changed or been impacted as the result of their participation. Through this type of data measure, you provide students the opportunity to reflect on their own out-of-class learning, which has the potential to support, or maybe even refute, your own findings within the data. Some examples of final data measures include:
- Final reflection writing activities,
- A survey with Likert-scale items, open-ended prompts or both that revisit initial survey questions or ask completely new ones,
- A focus group with your students,
- Interviews with student participants,
- A final project report or summary,
- A presentation of the work that was carried out,
There is no doubt that SoTL provides a unique opportunity for researching and assessing out-of-class learning. Given that many university classes across the disciplines offer field experiences, service learning activities or special projects that take students out of the classroom in an effort to make real-world connections between content and practice, this kind of research can be used to provide insight into what students actually take away, both personally and professionally, from such experiences. However, in order to do this, SoTL researchers must be purposeful in how they not only design the project or experience in which students engage, but also how they will capture data at the beginning, end and at various points throughout.