The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Updated Advice for New SoTL Researchers

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences & Disorders at Illinois State University

Screen Shot 2018-06-23 at 6.48.06 PM.pngAlmost three years ago, I penned a blog post titled Advice for New SoTL Researchers. In that post, I offered seven suggestions for those just getting started with a SoTL research agenda. In the last few months, I’ve had the good fortune to work with several cohorts of faculty and students who are part of “Intro to SoTL” cohorts. In working with them, I realize that the advice I offer to new researchers has changed. So, the following is my best effort at updating a list of things that new SoTL scholars might want to keep in mind.

  1. Design your project carefully. Examine the macro (classroom-level) context around you, looking for problems, opportunities, or wonderments that might be the basis for a SoTL project. Do you have a new technology that you’re wanting to integrate in your class, but aren’t sure it will work? Are you teaching an evening section of a very large class and you have an idea about improving student engagement? Might there be a way to study an out-of-class learning experience you’ve set up for your students? All of these – and many others! – could be a great place to start!
  2. Once you have a glimmer of an idea of the topic you might like to study, search for teaching and learning research in your field or another that might demonstrate how your topic has been studied in the past. Because SoTL research functions to provide a snapshot of your teaching/learning context at a point in time, it is fine to replicate a project that has already been done to see if similar outcomes are evident in your context. That said, reviewing past literature might drive you in a different research direction or provide an idea of how other scholars have approached research design in the past.
  3. Talk to a person who has completed a SoTL project and ask for advice or consultation. I have found individuals involved in teaching and learning research to be some of the most giving and collaborative colleagues I’ve encountered. Most would be quite happy to share lessons learned or chat about your idea(s) for a project. Seek out experienced SoTL scholars and learn with and from them. Then, when it’s your turn to be the experienced mentor, offer your wisdom often and broadly.
  4. Choose your data source wisely. There are SO many options in terms of potential data sources for SoTL work. As we are studying teaching and learning, SoTL scholars frequently use class artifacts, assessments, or reflections as a source for data. Other surveys, interviews, or focus groups beyond the typical business of your course might be useful. You are only limited by your own lack of creativity here. Carefully asses the focus of your project to suss out the richest sources of data for your study. Think about direct vs. indirect sources and the impact of your data on the overall rigor and quality of your work. Identifying a data source for your work should not be a quick decision, but rather, a careful deliberation.
  5. Consider more than one data source. As there are inherent biases in SoTL (e.g., it’s not meant to be inherently generalizable in most cases, we study our own students, true randomization or control is hard to exert), it’s optimal to have at least two data sources to compare and contrast to help validate the conclusions that you draw.
  6. Analyze and interpret your data appropriately. This piece of advice likely doesn’t need a lot of explanation; however, I would simply offer that you should think carefully about whether a qualitative, quantitative, or mixed methods approach to data analysis and interpretation is best for your corpus of data. Don’t force a fit, just finesse what you have so that the path to understanding your teaching/learning question is clear.
  7. Think about the audiences most suited for your work as you plan to share it with peers and others. Don’t assume that the potential audience for your work is broad and cross-disciplinary if your project only studies a phenomenon that is part of your discipline. Conversely, if your SoTL project focuses on a topic that has multi-disciplinary appeal, don’t narrow your audience unnecessarily. Share, publish, and promote your work in meaningful contexts with the individuals who will find it valuable!
  8. Put students at the heart of your SoTL. It has been well-stated that the heart of SoTL is the classroom. I choose to interpret this sentiment as not just a reminder that the single classroom context is the typical and intended focus of SoTL. Rather, I believe that the heart of SoTL subsumes the entire classroom environment and all the stakeholders within. Yes, you may study your students as research participants, but does that preclude you from sharing what you’ve learned with them? That is an opportunity that is often missed, in my view. Also, why not invite students to assist with your SoTL with the same frequency that you invite them into your disciplinary research? From my experience, it’s valuable for your students AND for you.

Of course, this is in NO way an exhaustive list of recommendations for new SoTL researchers. What is represented here is a continued starting place, on that will likely continue to evolve. Maybe three years from now, I’ll feel obligated/motivated to revise this list again! Until then, happy SoTL-ing! J

Advertisements


Leave a comment

Giving the Reading of SoTL Impact

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

On my flight home from a conference in sunny Austin, Texas last week (as I type this it’s snowing in Illinois, so the “sunny” descriptor is a happy recollection!), I had the opportunity to catch up on some journal reading that had accumulated. One piece I was interested in reading was an editorial from the most recent issue of InSight: A Journal of Scholarly Teaching. Written by Nancy Chick, this work (titled Does Reading SoTL Matter? Difficult Questions of Impact) discussed the issue of impact in SoTL and questioned the influence of reading SoTL on a practitioner’s teaching and on student learning. In doing so, Chick raised a troubling question in the minds of her readers: what if reading SoTL doesn’t lead to any change in teaching or learning practice? I’m fairly certain that SoTL researchers don’t produce their work to have it NOT inform future teaching and learning practices. So, are we missing the “application” boat where we take what we read and use it to solve teaching and learning problems?

readI hate to think that SoTL reflects the trend identified in medical fields (“journals are not good at getting doctors to change and improve their practice”). However, I do feel as though the impact of reading SoTL research could easily be diminished without some sort of purposeful process of reflection, discussion, and/or integration – in the same manner that research says our students learn new skills. What might that look like, though? Chick suggests several wonderful options (a SoTL Journal Club, the use of small networks to discuss SoTL, and greater access to SoTL research via open access mechanisms to make discussions about our SoTL readings possible).

The overarching suggestion in this article was that those of us who read SoTL should “talk with others about what these readings make [us] think about.” I agree, for in that practice, there IS impact. Honestly, think about it. If you read SoTL research and then engage in discussions about what you’ve learned with others, you (very likely) consider your readings more deeply and puzzle over application of the study’s outcomes more thoroughly. Sharing leads to a deeper understanding — and perhaps use — of what we’ve read.

After reading Chick’s article, I spent the remainder of my plane ride thinking about other ways in which conversations about our own SoTL readings might be encouraged –beyond those suggested in the article. I have a few suggestions, across a variety of stakeholder groups/levels. These look a lot like general advocacy suggestions for SoTL, though each is tied to the specific practice of reading SoTL, with subsequent advocacy (aka: sharing) building impact over time:

  • Help peers develop an awareness of SoTL. If they don’t know a body of research about teaching and learning exists, they will never attempt to read it! Share resources where evidence on teaching and learning can routinely be accessed. Explain – explicitly — how you’ve used SoTL readings to alter your teaching practice(s). Take it one step further and detail how reading SoTL led you to conduct your own SoTL study.
  • Seek out formal and informal ways to share new knowledge derived from reading SoTL with colleagues or other stakeholders such as students, department or campus administrators, disciplinary leaders, and/or community members. Summarize what you’ve learned in newsletters, staff meetings, emails…any communication mechanism that allows for an exchange of this information. Approach your institution’s teaching and learning center to suggest programming based around reading SoTL to inform a scholarly approach to teaching.
  • Mentor students in reading and applying SoTL research. Share insights about learning with students to help them develop scholarly approaches to learning as well as scholarly approaches to teaching.
  • Add value to what you share with campus administrators about the SoTL you read by tying new knowledge from your SoTL readings to updates to the mission/vision of the institution or to its strategic plan. Advocate for evidence-informed thinking about next steps for your campus.
  • Use social media to share summaries of SoTL research with relevant stakeholders. Give an overview of what you read, then provide a link to the primary source for further exploration. Ask questions to encourage discussion among your “followers” to further develop ideas related to your SoTL readings.
  • Network at conferences to share case studies of how reading SoTL research has led to pedagogical change. This is particularly important at disciplinary conferences as widespread understanding of SoTL research is less obvious in those contexts than is typically evident at a teaching/learning conference.

These ideas in no way constitute an exhaustive list! Please feel free to add suggestions from your own context/practice below in the comments section! Happy SoTL reading – and sharing!

Blog References

Chick, N. L. (2017). Does reading SoTL matter? Difficult questions of impact. InSight: A Journal of Scholarly Teaching, 12, 9-13.

 

 

 


1 Comment

Problems, Opportunities, and Wonderments: Possible Subsets of “What works?”

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences & Disorders at Illinois State University

what worksWhen I talk to people new to SoTL — students, faculty, other interested folks — I am sometimes asked what a “typical” SoTL research question might be. Part of my description of SoTL details that a fair amount of SoTL is context-specific and is meant to gather information or data about a fairly restricted population: the students (or teachers) in the course or experience being studied. I explain that while SoTL is not inherently generalizable, if enough people in enough different contexts study similar questions, we can develop standards for high-impact teaching and learning practices that CAN transfer across classrooms, disciplines, and/or institutions. Kuh (2008) wrote of high-impact practices for undergraduate education that exemplify this idea very nicely.

But back to those “typical” SoTL questions…it’s not always easy for those new to SoTL to identify one thing to study in their first SoTL project. In Opening Lines: Approaches to the Scholarship of Teaching and Learning, Pat Hutchings (2000) laid out a wonderful taxonomy of questions to characterize the main foci evident in most scholarship of teaching and learning work: what works, what is, visions of the possible, and theory building questions. She describes “what works?” questions as being the typical starting place for most new SoTL researchers, as topics falling into this part of Hutchings’ taxonomy focus on investigating the effects of different approaches to teaching and learning. Having facilitated numerous “Intro to SoTL” experiences for faculty and students, I wholeheartedly agree with Hutchings that this “what works?” level of inquiry often is the first explored by new SoTL scholars. It is on that level of Hutchings’ taxonomy that I focus my thoughts today.

I have found that it’s helpful to break down “what works?” concepts into three subcategories in an effort to encompass possible areas of SoTL study. I use the following terms, though many others could be substituted easily:

  • Problems exist in the teaching and learning contexts of most instructors, and typically involve doubt, insecurity, or difficulty in some form or fashion. Problems exist when any aspects of classroom environment, course content, or course management cause trouble for students or for the course instructor. Potential problems that could be studied in a SoTL project include:
    • determining how to use problematic classroom space most effectively,
    • managing active learning with large course enrollment,
    • figuring out why a particular class/lab/experience seems to be very difficult for students.
  • Opportunities are variables that become a part of your learning context, whether you placed them there or they occur via happenstance. These are usually perceived by students and course instructors as more positive in nature than are problems. Opportunities that might be studied as part of a SoTL project include:
    • identifying the impact of a study abroad experience,
    • measuring the differences between flipped and traditional teaching designs,
    • analyzing student learning as a result of a service learning associated with a particular course.
  • Wonderments* lead to pedagogies that are integrated into a course/learning context in a creative manner. Wonderments begin with the question “what would happen if we did ___________?” adding something that otherwise wouldn’t exist in a course to address an instructor-conceived idea. Examples of potential wonderments that might be studied in a teaching/learning context are:
    • implementing pre-course modules designed to decrease math anxiety for students in a chemistry course,
    • using arts-based observation methods to help doctors, nurses, or other clinical professionals be more effective diagnosticians,
    • creating a new pedagogy (or merging others together) to see if they support student learning (e.g., combining case study teaching with perspective-taking to encourage students to understand clinical cases more comprehensively).

The examples above are obviously not exhaustive, but are meant to illustrate each of these terms as I use them. Is it possible that overlap exists across the categories of problems, opportunities, and wonderments? I would think so, particularly in terms of wonderments, as a creative idea might be used in addressing a problem or in creating an opportunity. That said, as wonderments occur on their own as well, I thought them to be deserving of their own descriptor.

Why do these possible subcategories of Hutchings’ “what works?” question matter? I have consistently found that using subcategories makes it easier for new SoTL researchers to identify the focus of their first study – and to understand why they are interested in studying that topic. As a SoTL faculty developer, anything that facilitates research on teaching and learning and helps crystalize ideas about SoTL is something worth using!

*The term wonderment was inspired by Dr. Ken Jerich, my dissertation adviser, who regularly used this term in his teaching and research. Obviously, I do now, as well.

 

Blog References:

Hutchings, P. (2000). Opening lines: Approaches to the scholarship of teaching and learning. Carnegie Foundation for the Advancement of Teaching: Menlo Park, CA.

Kuh, G. (2008). High-impact educational practices: What they are, who has access to them, and why they matter. AAC&U: Washington, DC.


1 Comment

Prospective Students and Parents: An Opportunity for Macro-level SoTL Advocacy?

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

considerRecently, I have started to wonder if we, proponents of and for SoTL, might be missing an opportunity to connect with an important group of stakeholders as part of our “typical” SoTL advocacy. We regularly and routinely share the value and importance of SoTL with faculty and campus administrators. We advocate within our disciplines and across our institutions. Conversations at SoTL conferences have focused – rightly so – on the lack of student voices in our SoTL work. So many SoTL folks now strongly advocate for students to be partners in our SoTL endeavors. These are important, impactful efforts to continue building SoTL and likely always will be! That said, I think many of us are leaving an important group — prospective students and parents — out of our campus-level (macro) SoTL advocacy.

I will admit to having a unique perspective on this topic: my son is currently a high school junior. We have looked at numerous college websites and have visited half a dozen colleges. It was during one of these visits that my son asked a faculty member he met, “do you have the chance to do SoTL research here?” I was surprised by his question. I had been wondering the same thing, but figured that was simply because SoTL is my professional passion. I hadn’t stopped to consider that my son might care about this, too. Prospective students might really benefit from knowing that a university supports the study of student learning to improve teaching. For prospective parents, this might be equally important to inform discussions and priorities related to college choice.

At a time when the general societal attitudes are not always kind to higher education, it may be truly valuable that we demonstrate to prospective students and parents that there is meaningful research being conducted on student learning that is meaningful in the context of our individual institutions. Sharing how this research can improve the student experience at a university might help these stakeholders make important choices based not on “brand,” but rather on substance.

What mechanisms could be utilized to support the sharing of SoTL work with prospective students and parents at your university? I offer several suggestions below, though this is hardly an exhaustive list!

  • Provide information (perhaps linked on your institution’s admissions website) about SoTL on your campus. Highlight the work of faculty and students. EXPLAIN why SoTL matters!
  • Record and report testimonials on the impact of SoTL for students on your campus website. Specifically describe how course instructors use or apply SoTL to improve student learning. SHOW how SoTL makes an impact.
  • It might be even more important to include information about SoTL accomplishments on specific department/unit websites. Reports have shown that the most common web searches engaged in by prospective students and parents are specific to majors/minors/academic programs than any other. CONTEXTUALIZE discipline-specific SoTL work.
  • Think about how social media is used on your campus. I follow the Instagram and Twitter feeds from my son’s “top five” universities. It’s remarkable how much you can learn about what a university values just by doing this! Sadly, I’ve very rarely seen posts about student learning or SoTL from these accounts, though such posts would be very appropriate, and helpful. Think about how you can work with your institution’s social media managers to reach prospective students and parents through accounts such as these to advocate for the SoTL being done on your campus. INTEGRATE SoTL into your institution’s public image.
  • Encourage admissions officers and other campus social media managers to share information relative to SoTL news and accomplishments on your campus. ADVOCATE for SoTL work to be shared.
  • Social media/websites aren’t the only way to reach prospective parents and students, though they are likely the most common. Identify mechanisms at your institution that could be useful in sharing information about SoTL to this group of stakeholders. Perhaps an alumni magazine, community publication, or other outlet exists where information about SoTL can be shared. CONSIDER the possibilities for sharing SoTL in print and via other media.

 


Leave a comment

Photo Documentation: SoTL Methods Series #4

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

Over the last five years, students from my academic department have traveled to six countries on two continents as part of our program’s study abroad experience. Offered for independent study credit, these study abroad experiences have transpired as “short term” opportunities (e.g., spring break or two weeks in the summer session) and have focused on cultural immersion, rather than disciplinary content knowledge. I led our most recent trip to Spain during spring break 2017 and was joined by two faculty colleagues and 32 students. Prior to this year’s trip, students self-reported gains in what Miller-Perrin and Thompson (2014) would consider internal learning (e.g., emotional, intellectual, and spiritual growth) as well as external learning (e.g., second language acquisition, intercultural learning, globalization, disciplinary knowledge). While this was lovely to hear, these impressions were anecdotal, as we had never systematically studied student learning that occurred as a result of engagement as a study abroad participant.

I decided to study this year’s students to measure the impact of short term study abroad for this cohort. Initially, I thought I would restrict my study design to traditional methods of reflection and content analysis. In the weeks prior to travel, I asked students to set three goals for themselves based on self-perceived areas of weakness or interest. Students were asked to keep track of their growth in these goal areas via reflective journals, which were analyzed carefully for evidence of progress or growth in their goal areas. Eighty-one of these goals fit into one of the following categories: taking chances, engagement, flexibility, gaining independence, archiving, budgeting, social interaction, or understanding culture.

Around the time that I was developing the plan for collecting data from travelers, I read about photo documentation, a method derived from visual sociology wherein researchers seek out patterns in photographic data collected using something called a shooting script. I was intrigued as to how I might introduce this new method into my study abroad project. I tried, and I was able to extend what I learned about my students in doing so. The remainder of this post is a description of my “rookie experience” with photo documentation. I am certainly not an expert, but I did enjoy exploring this visual methodology!

Photo documentation is a research method developed by Charles Suchar (1997). Specifically, “photo documentation is a method that assumes photographs are accurate records of what was in front of the camera when its shutter snapped – ‘a precise record of material reality’ — and takes photographs in a systematic way in order to provide data which the researcher then analyzes” (Rose, 2016, p. 310). The key to photo documentation is a shooting script which consists of “lists of sub-questions” (derived as topical to overarching research questions) which act as a guide for taking pictures connected to the topic of the research being conducted (Rose, 2016, p. 311). Photos are taken in accordance with a shooting script, then analyzed for categories and patterns via a systematic coding process.

In the case of the study I described above, I wanted to understand changes that occurred in students in self-identified areas of need/interest. Thus, in addition to reflective (written) journaling, I asked that students use the following shooting script to document learning visually before and during travel:

  1. What was I doing/seeing when I recognized that I had made progress toward meeting one of my goals?
  2. What things/people/experiences influenced this progress?

spain food 2In applying this shooting script, students were directed to take pictures to answer these questions as they were going about their study abroad experiences. After travel, they submitted three photos per goal to me, along with their written journals. I printed out all photos that students submitted and searched for patterns across goals and photos. I was able to find many! As an example, many students set goals to improve their understanding of different cultures. Looking at the photos submitted along with written reflection for this goal, I could see that students represented “increases in cultural understanding”  via photos of food (see right), religious symbols, architecture, or Spanish citizens they interacted with during their travels. Students who set goals to “take chances” maggierepresented growth in this area with photos of new foods they tried or photos of experiences where they conquered personal fears (e.g., communing with the macaques in Gibraltar after a prior bad experience — see left).

Overall, what I found interesting is the different stories these data told. While written reflections told me WHAT changes students had realized as a result of study abroad, the photographs told me HOW how these changes occurred. The combination of these different data types were powerful to tell the story of my students’ experiences in Spain and made a powerful case for the possibility of significant learning in a short amount of time.

I have always been intrigued with what the visual representation of learning might look like for different students. Using photo documentation helped me to see the possibilities of visually-based data in a way that I appreciate and hope to use again in the future!

Blog References:

Miller-Perrin, C. & Thompson, D. (2014). Outcomes of global education: External and internal change   associated with study abroad. New Directions for Student Services, 146, p. 77-89.

Rose, G. (2016). Visual methodologies: An introduction to researching with visual methods (4th ed.). Sage: Los Angeles.

Suchar, C. S. (1997). Grounding visual sociology research in shooting scripts. Qualitative Sociology, 20(1), 33-55.

 


Leave a comment

Musings on SoTL Peer Mentorship

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences & Disorders at Illinois State University

pointRecently, I worked with faculty at Bradley University to develop a framework for and guidance in SoTL peer mentoring. Bradley is working diligently to increase engagement in SoTL and have adopted a “grow their own” approach to this effort, selecting faculty who have been SoTL-productive to mentor other faculty members interested in becoming SoTL scholars. The process of preparing for this undertaking led me to (over time) merge my anecdotal experiences as a SoTL mentor with evidence about peer mentoring (in and out of SoTL). I’ve tried to organize some of these reflections below:

  • In preparing my session, I looked toward existing research on peer mentorship in SoTL, finding little. One study I did find was from Hubbal, Clark, and Poole (2010), who analyzed ten years of data on SoTL mentoring to identify three critical practices of SoTL mentors : modeling of SoTL productivity, facilitation of mentees’ SoTL research, and engagement in SoTL networking with other SoTL scholars. In terms of my SoTL mentee/mentor experiences, I think the last practice, that of connecting mentees with other SoTL scholars, is critical and often neglected. Introducing novice SoTL scholars to the “commons” of SoTL has the potential to sustain interest, broaden perspectives, and increase engagement in the SoTL movement as a whole.
  • Often times, when I do “intro” workshops to explain SoTL to new students and faculty, there is a perception that SoTL research is very different from disciplinary research. I always explain that while it can be, it really isn’t in many ways! Similarly, I have found that faculty who have extensive disciplinary experience mentoring students and peers struggle to understand that SoTL mentorship really isn’t all that different. The same practices applied to a differently-focused research project can be very successful in helping a novice SoTL researcher gain confidence in conducting research on teaching and learning.
  • Zellers, Howard, and Barcic (2008) found that benefits to mentees engaged in mentorship programs included assimilation to campus culture, higher career satisfaction, higher rate of promotion, and increased motivation to mentor others. While this work was not focused on SoTL, I can easily see how the same tenets might apply to research on teaching and learning, as well. In terms of SoTL research, I’d add that benefits could include opportunities for assimilation to SoTL culture at and beyond the single institutional level as well as the chance to work with mentors and faculty across varied fields of study in a way that isn’t always customary in disciplinary research.
  • Clutterbuck and Lane (2016, xvi) state “to some extent the definition of mentoring does not matter greatly, if those in the role of mentor and mentee have a clear and mutual understanding of what is expected of them and what they should, in turn, expect of their mentoring partner.” This is so true! The most successful peer mentoring relationships I’ve witnesses have strong foundations in clear and regular communication of expectations, progress, bottlenecks, etc.
  • I’ve encountered two types of SoTL peer mentorship frameworks: formal (set framework for participation and, often, assignment of mentor/mentee pairs) and informal (relationships that develop by happenstance due to opportunity and shared interests). I feel that there are likely benefits to each. Formal mentorship programs are more likely to have stronger administrative support and integration of the program within a strategy for professional development, both characteristics of successful mentoring programs (Hanover Research, 2014). Conversely, informal peer mentoring frameworks allow for voluntary participation, participant involvement in the mentor/mentee pairing process, and the ability for participants to co-develop goals, expectations, and desired outcomes of the mentorship paring, each also components of successful mentoring programs (Hanover Research, 2014). So, which is better and why? This might be a very interesting area for future study, as currently, we just don’t know.
  • What makes a successful peer mentor? Awareness of adult learning principles/teaching strategies/techniques, and understanding/acknowledgement of differences in orientation and stage of development between themselves and their mentees, and ability to plan/observe/facilitate discussion (Knippelmeyer & Torraco, 2007). It would seem that many folks engaged in SoTL, then, would make excellent peer mentors, as these characteristics are as endemic to SoTL as they are to mentorship!

Blog References:

Clutterbuck, D. & Lane, G. (2016). The situational mentor: An international review of competences and capabilities in mentoring. London: Routledge.

Hanover Research. (2014). Faculty mentoring models and effective practices. Author.

Hubball, H., Clarke, A., & Poole, G. (2010). Ten-year reflections on mentoring SoTL research in a research-intensive university. International Journal for Academic Development, 15(2), 117-129.

Knippelmeyer, S. A. & Torraco, R. J. (2007). Mentoring as a developmental tool for higher education. University of Nebraska-Lincoln teaching center publication.

Zellers, D. F., Howard, V. M., Barcic, M. A. (2008). Faculty mentoring programs: Reenvisioning rather than reinventing the wheel. Review of Educational Research, 78(3), 552-588.

 


Leave a comment

College Rankings, Student Learning, and SoTL : An Unlikely Trio?

Written by Jennifer Friberg, Cross Endowed Chair in the Scholarship of Teaching and Learning, and Associate Professor of Communication Sciences and Disorders at Illinois State University

ratingLast Friday (9/8/2017), the Chronicle of Higher Education published an interesting article written by Richard M. Freeland titled “Stop Looking at Rankings. Use Academe’s Own Measures Instead.” Ostensibly, this piece discusses the role and utility of college rankings such as those published annually by U.S. News and World Report. Freeland explains that there are some measures reported in these rankings that are “legitimate indicators of academic quality,” such as “graduation rates, faculty qualifications, and investment in academic programs.” He goes on to say that other rankings (the federal government’s College Scorecard, extant Integrated Postsecondary Education System data, and the Voluntary System of Accountability) have added important data to the conversation of what makes a college “good” in a world where it’s hard to determine institutional quality. He is undeniably correct. However, for years I have felt as though we have been missing the boat with our current reporting of college rankings, as we seem to in no way account for student learning as part of these metrics. Due in large part to my background in teaching and learning (and my status as the parent of a high school junior seeking a future university home!), this is very frustrating to me, for I want to know more about student learning outcomes at institutions than about many other data points. This is a huge void and something I’ve considered an opportunity for SoTL for a long, long time.

Freeland writes of a “deep resistance within academe to publishing data about what students learn,” providing a historical overview of various standardized measures of intellectual achievement that have been proposed – and rejected — as universal measurements of student learning. And, so, the void in the reporting of student learning as a important data point in college rankings remains. Freeland remarks:

While many colleges have developed programs to assess student learning (often because of accreditation requirements), few systematically collect and even fewer publish quantitative data that allow readers to compare student intellectual achievement across institutional lines. Until this gap is filled, higher education’s systems of accountability will continue to be data-rich but information-poor with respect to the quality to actual learning. The public will be left to rely on commercial rankings as indicators of institutional quality.

Based on all of this, my overarching question is this: as advocates for the scholarship of teaching and learning – the very research that CAN help provide information about student learning in higher education to the public – is it important that we promote SoTL as a potential valuable contributor to the college rankings discussion?

I’ve struggled with this question all weekend. Here’s where my thinking is at this point:

  • I believe that SoTL does belong in the college ranking discussion. Student learning is our wheelhouse. We need a seat at this table to advocate for and honor outcomes of a diverse and rich field of scholarship on student learning. SoTLists cannot allow student learning to be assessed via a standardized test or any other “one look” measure of performance and expect to tell the whole story. Scholars of teaching and learning recognize this well and can advocate accordingly.
  • I don’t believe that student learning should be ranked. I can see dangers in how this could happen if we start talking about which school has better learning outcomes than others might. Student learning varies by context and discipline, creating a number of limitations on “best learning outcome” data that could be reported. There is no universal curriculum for higher education. As such, any ranking system of student learning would lack reliability and validity.
  • Years and years of research on teaching and learning has led to the understanding of high-impact practices for undergraduate education, and more such information is shared regularly in cross-disciplinary and discipline-specific publications.

So, then, perhaps what we are looking to capture in college rankings shouldn’t specifically focus on student learning outcomes. Every institution of higher education is looking to support student learning, but we must acknowledge that this is accomplished in a manner that honors contextual differences as well as institutional missions and ideologies. With this in mind, any comparison of student learning across institutions may well be akin to comparing apples to oranges.

That said, you CAN measure the use of evidence-based approaches in higher education (e.g., undergraduate research, service learning/community-based learning, internships, and first-year seminars and experiences; Kuh, 2008) to measure quality of instruction. SoTL scholars add to the evidence-based for scholarly learning daily. We have solid evidence that certain instructional methods work and data could be collected to reflect how often these pedagogies are used in college classrooms. That might be a metric of interest to various stakeholders. I’m confident there are others, as well. I’m still stewing on this and am curious what others are thinking. I think this is an important discussion and one that we, as SoTL advocates, should be a part of, to explore this idea some more. To that end, feel free to comment below or continue the discussion on social media (@ISU_SoTL).

Blog References:

Freeland, R. M., (2017, September 8). Stop looking at rankings. Use academe’s own measures instead. Chronicle of Higher Education. Downloaded from: http://www.chronicle.com/article/Stop-Looking-at-Rankings-Use/241140?cid=at&utm_source=at&utm_medium=en&elqTrackId=4eadfa107c984352bd8664bf86cba24d&elq=869cd22487394fe88b84eb2d7904a1d2&elqaid=15516&elqat=1&elqCampaignId=6640#comments-anchor

Kuh, G. D. (2008). High-impact educational practices: Why they are, who has access to them, and why they matter. Washington, DC: AAC&U.

McKinney, K. (n.d.). A sampling of what we know about learning from scholarship of teaching and learning and educational research. Downloaded from: http://sotl.illinoisstate.edu/downloads/materials/A%20Sampling%20of%20What%20We%20Know.pdf