Written by Jennifer Friberg, Cross Endowed Chair in the Scholarship of Teaching and Learning, and Associate Professor of Communication Sciences and Disorders at Illinois State University
Last Friday (9/8/2017), the Chronicle of Higher Education published an interesting article written by Richard M. Freeland titled “Stop Looking at Rankings. Use Academe’s Own Measures Instead.” Ostensibly, this piece discusses the role and utility of college rankings such as those published annually by U.S. News and World Report. Freeland explains that there are some measures reported in these rankings that are “legitimate indicators of academic quality,” such as “graduation rates, faculty qualifications, and investment in academic programs.” He goes on to say that other rankings (the federal government’s College Scorecard, extant Integrated Postsecondary Education System data, and the Voluntary System of Accountability) have added important data to the conversation of what makes a college “good” in a world where it’s hard to determine institutional quality. He is undeniably correct. However, for years I have felt as though we have been missing the boat with our current reporting of college rankings, as we seem to in no way account for student learning as part of these metrics. Due in large part to my background in teaching and learning (and my status as the parent of a high school junior seeking a future university home!), this is very frustrating to me, for I want to know more about student learning outcomes at institutions than about many other data points. This is a huge void and something I’ve considered an opportunity for SoTL for a long, long time.
Freeland writes of a “deep resistance within academe to publishing data about what students learn,” providing a historical overview of various standardized measures of intellectual achievement that have been proposed – and rejected — as universal measurements of student learning. And, so, the void in the reporting of student learning as a important data point in college rankings remains. Freeland remarks:
While many colleges have developed programs to assess student learning (often because of accreditation requirements), few systematically collect and even fewer publish quantitative data that allow readers to compare student intellectual achievement across institutional lines. Until this gap is filled, higher education’s systems of accountability will continue to be data-rich but information-poor with respect to the quality to actual learning. The public will be left to rely on commercial rankings as indicators of institutional quality.
Based on all of this, my overarching question is this: as advocates for the scholarship of teaching and learning – the very research that CAN help provide information about student learning in higher education to the public – is it important that we promote SoTL as a potential valuable contributor to the college rankings discussion?
I’ve struggled with this question all weekend. Here’s where my thinking is at this point:
- I believe that SoTL does belong in the college ranking discussion. Student learning is our wheelhouse. We need a seat at this table to advocate for and honor outcomes of a diverse and rich field of scholarship on student learning. SoTLists cannot allow student learning to be assessed via a standardized test or any other “one look” measure of performance and expect to tell the whole story. Scholars of teaching and learning recognize this well and can advocate accordingly.
- I don’t believe that student learning should be ranked. I can see dangers in how this could happen if we start talking about which school has better learning outcomes than others might. Student learning varies by context and discipline, creating a number of limitations on “best learning outcome” data that could be reported. There is no universal curriculum for higher education. As such, any ranking system of student learning would lack reliability and validity.
- Years and years of research on teaching and learning has led to the understanding of high-impact practices for undergraduate education, and more such information is shared regularly in cross-disciplinary and discipline-specific publications.
So, then, perhaps what we are looking to capture in college rankings shouldn’t specifically focus on student learning outcomes. Every institution of higher education is looking to support student learning, but we must acknowledge that this is accomplished in a manner that honors contextual differences as well as institutional missions and ideologies. With this in mind, any comparison of student learning across institutions may well be akin to comparing apples to oranges.
That said, you CAN measure the use of evidence-based approaches in higher education (e.g., undergraduate research, service learning/community-based learning, internships, and first-year seminars and experiences; Kuh, 2008) to measure quality of instruction. SoTL scholars add to the evidence-based for scholarly learning daily. We have solid evidence that certain instructional methods work and data could be collected to reflect how often these pedagogies are used in college classrooms. That might be a metric of interest to various stakeholders. I’m confident there are others, as well. I’m still stewing on this and am curious what others are thinking. I think this is an important discussion and one that we, as SoTL advocates, should be a part of, to explore this idea some more. To that end, feel free to comment below or continue the discussion on social media (@ISU_SoTL).
Freeland, R. M., (2017, September 8). Stop looking at rankings. Use academe’s own measures instead. Chronicle of Higher Education. Downloaded from: http://www.chronicle.com/article/Stop-Looking-at-Rankings-Use/241140?cid=at&utm_source=at&utm_medium=en&elqTrackId=4eadfa107c984352bd8664bf86cba24d&elq=869cd22487394fe88b84eb2d7904a1d2&elqaid=15516&elqat=1&elqCampaignId=6640#comments-anchor
Kuh, G. D. (2008). High-impact educational practices: Why they are, who has access to them, and why they matter. Washington, DC: AAC&U.
McKinney, K. (n.d.). A sampling of what we know about learning from scholarship of teaching and learning and educational research. Downloaded from: http://sotl.illinoisstate.edu/downloads/materials/A%20Sampling%20of%20What%20We%20Know.pdf