The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…

Leave a comment

Does College Teach Critical Thinking?

Written by Jen Friberg, SoTL Scholar-Mentor at Illinois State University

Last week, the Chronicle of Higher Education’s blog published a post titled, “Yes, Colleges Do Teach Critical Thinking Skills.” This article reviewed work by Christopher Huber and Nathan Kuncel, who undertook a large-scale meta-analysis on 71 papers published in the last 50 years that reported on changes in critical thinking in college students.

What Huber and Kuncel found was interesting: students’ critical thinking skills DO improve in college. Yet, students seem to develop these skills with or without explicit education in how to be a critical thinker. Beyond this finding, Huber and Kuncel reported that:

  • College students do make measurable and substantial gains in critical thinking during their undergraduate studies.
  • There is no “unequivocal” support for the notion of an “acceleration effect,” wherein students learn more critical thinking skills in the later years of their undergraduate experience.
  • Those interested in studying their students’ critical thinking skills should carefully consider their methodology for doing so, as effect size differences were evident in cross-sectional vs. longitudinal research designs.
  • Observed gains in critical thinking skills seem to have decreased over time, despite increased emphasis on critical thinking pedagogies.
  • College fosters increases in critical thinking disposition, meaning that students identify and appreciate the need to be critical thinkers in different life contexts.
  • When considering all findings together, teaching domain-general critical thinking skills may not be the best use of classroom resources.

Despite these findings, questions remain about critical thinking. If students are learning about critical thinking in college, and that learning is not attributable to coursework focused on critical thinking, where are students learning this skill? Does maturation and experience come into play? If so, how? Do internships or other clinical experiences impact development of critical thinking? What influence might Interdisciplinary education have? Are we measuring critical thinking skills in a valid manner? Should critical thinking be measured differently according to disciplinary teachings/emphases? Yes, many questions about critical thinking still remain!

Have you measured critical thinking in your students? What methods did you use? What did you find out? Can you link critical thinking directly to a course or pedagogical choice? We’d love to hear about your experiences here! Please comment below to share.


Call for Chapters: Applying the Scholarship of Teaching and Learning Beyond the Individual Classroom Level

We invite our blog readers to consider submitting a chapter proposal or to share this call with colleagues for a planned SoTL text:

Applying the Scholarship of Teaching and Learning Beyond the Individual Classroom Level

Jennifer Friberg and Kathleen McKinney, Editors, Illinois State University

We are seeking brief chapter proposals for editorial review and possible inclusion in this volume. The purpose of this book is to share with others in higher education examples of applying scholarship of teaching and learning (SoTL) results to teaching and student learning beyond the individual classroom level. The focus is on the use of SoTL–practitioner, action reflection/research (usually on the researchers’ own students and/or students in their discipline at a local level)–and its application at the multi-section course, program, department, college, co-curricular, institutional, or disciplinary level(s). The SoTL results or implications applied could be from the author’s own SoTL project(s) and/or from application of a synthesis of others’ SoTL work on a given topic or in the discipline.

We are seeking expert authors with high quality ideas from diverse disciplines, institutions, and nations to propose chapters that discuss such an application. Proposals should include:

  • a description of the SoTL project(s) and literature from which it came
  • strategies, mechanisms or processes used to apply SoTL results
  • outcomes or future implications
  • lessons learned in applying SoTL and/or advice for others seeking to apply SoTL
  • other novel ideas on the application of SoTL beyond the individual classroom level.

To submit, send a one to two-page chapter proposal summarizing the ideas for your chapter as they fit the requirements above by February 15, 2016 to and Our tentative time frame is to select chapters/authors by April 2016 and immediately submit our book proposal to multiple publishing companies. As soon as a book contract is obtained, authors will be notified and given about 4-5 months to submit the chapter for editorial review, followed by about 2-3 months for revisions.

Editors: Dr. Jennifer Friberg is an Associate Professor of Communication Sciences and Disorders and SoTL Scholar-Mentor at Illinois State University. Friberg has been active in her disciplinary SoTL movement though her research and service to the American Speech-Language-Hearing Association and the Council on Academic Programs in Communication Sciences and Disorders. She is the co-author of the first text focused on SoTL in her profession: Scholarship of Teaching and Learning in Speech-Language Pathology and Audiology: Evidence-Based Education, and editor of The SoTL Advocate Weblog. Dr. Kathleen McKinney is Professor Emeritus of Sociology and the Cross Chair in the Scholarship of Teaching and Learning at Illinois State University. McKinney was a 2003-2004 Carnegie SoTL Scholar and served three years as editor of Teaching Sociology. She has been active in the SoTL movement locally, nationally, and internationally for decades doing her own SoTL research, engaging in service to the field of SoTL, and assisting others with their SoTL work. She has published two other books on SoTL: Enhancing Learning through the Scholarship of Teaching and Learning: The Challenges and Joys of Juggling and The Scholarship of Teaching and Learning In and Across the Disciplines.

Leave a comment

Three Key Teaching Questions & Our Obligation to Follow the Research

Written by Jennifer Friberg, SoTL Scholar-Mentor at Illinois State University

Last week, I had the good fortune to attend the 45th annual conference of the International Society for Exploring Teaching and Learning (ISETL) in Savannah, GA. While the weather was delightful, the ideas discussed among attendees were even better! Now back at home and work, I’ve been busy reflecting on my experiences over the three days of the conference. While several blog posts will be posted in the coming weeks that originate in ideas generated and/or shared at ISETL, I wanted to first reflect on ideas from the keynote address delivered at the opening of the conference by Dr. Terry Doyle (titled: A New Paradigm for Student Learners). While this address focused on interesting research-based necessities to support student learning (hydration, sleep, exercise, and diet), it was the initial content of the address that provided a foundation for my thinking about teaching and learning ever since.


Doyle asked the audience to ponder what he termed Three Key Teaching Questions:

  1. What would make us happy that our students still knew and could apply from the content and skills of our course a year later?
  2. What knowledge and skills do students need our help with and what can they do on their own?
  3. What teaching actions optimize the opportunities for students to master the learning outcomes of our courses?

He very clearly indicated that in answering these questions, we, as course instructors, are obligated to follow where research takes us (even if it’s not comfortable to do so!), which immediately resonated with my interest in the application and generation of SoTL research. As course instructors, we regularly engage in reflection on the classes we teach, asking ourselves questions such as:

  • Did my terminal project lead to transformative learning?
  • Did my students learn as much in collaborative group experiences as I hoped they might?
  • How does the content of my course compare with other, similar courses at other institutions?
  • Are my students truly engaged in learning course content?
  • Do students in my discipline learn differently than students in other disciplines?

These are all questions that we can take a scholarly approach to answering, using information in or across various disciplines to influence our pedagogical choices. We can examine published teaching and learning research for help and guidance. Or, we can create and answer our own research questions to address our wonderments about teaching and learning. Either way, we can answer Doyle’s three key teaching questions by following where the research takes us…and our students can ultimately benefit!


Using Think Alouds to Collect Data for Your SoTL Study

Written by Sarah M. Ginsberg, Ed.D., Professor at Eastern Michigan University

A common thread that runs through various cross-disciplinary SOTL research is the concerted effort made to understand what the accomplished professional is thinking when she solves a work problem so that we can use that knowledge as teachers to better prepare future professionals. That problem might be how a mathematician completes a technical calculation, or in clinical fields, it might be how the clinician arrives at a diagnosis. The value for all of us in understanding what our accomplished colleagues do in their heads when faced with a technical problem is that in identifying how the pros do it, we can uncover insights into how we should be teaching our students to think and to problem solve. This type of understanding relies on a process of collecting data while the person is actively engaged in solving a problem out loud. This type of study is often referred to as a think aloud (TA) and can yield important information to inform evidence-based educational practices

The TA method is a validated method of learning about cognitive processes by having participants verbalize their thinking in a metacognitive manner (Ericcson & Simon, 1993; Wineberg, 1991). TAs were popularized by Wineberg (1991) in his ground-breaking study that examined the differences between how academic historians processed information while reading historical texts and how students processed information regarding historical texts.  Since then, TAs have been used to study how novice thinking compares to experienced thinking in a wide variety of disciplines, including the health sciences, mathematics, and political science (Banning, 2008; Bernstein, 2010; Forsberg, Ziegert, Hult, & Fors, 2013; Wainwright & McGinnis, 2009). These types of studies are often referred to as “expert-novice” studies (Bernstein, 2010).

The process of data collection using a TA approach is quite simple and requires minimal technology and cost. Typically:

  1. Study participants are presented with the problem to be solved by the researcher and asked to solve it aloud.
  2. Specific directions are provided to participants. Prompts (e.g., “tell me how you would solve this” or “describe how you would approach this problem”) are used to elicit responses and gather additional information if a participant falls silent or struggles with the process.
  3. Participant responses are recorded for subsequent transcription and analysis.
  4. Once the TA is transcribed, the most challenging part of the process becomes the subsequent data analysis. Consistent with qualitative methodology, verbalizations may be read as a whole to determine initial emerging codes and impressions about the thought process (Creswell, 2002; Denzin & Lincoln, 2012). Using an inductive approach to identifying specific thought processes or strategies allows the researcher to move forward to developing secondary, axial coding. Themes emerge as the iterative process expands to include all participants and commonalities and differences can be appreciated.

Having recently completed a study comparing the diagnostic problem-solving of experienced speech-language pathology (SLP) clinicians compared to the problem-solving of SLP graduate students, I found that the most challenging aspect of analyzing the data was determining the level of thinking to focus on. I used studies in comparable clinical professions, such as nursing, physicians, and physical therapists to identify frameworks that might be useful to me. In determining the focus of my study, I chose to concentrate on the heuristics (thinking strategies) of my participants, to understand differences in approaches to problem solving and to create a framework that fostered comparisons to previous literature, potentially increasing the value of my findings.

For more details on the think aloud method and some outstanding examples of its use in a variety of fields, see the items included in the following references. It should be noted that a number of authors also advocate for the use of TA as a teaching method. For those unfamiliar with qualitative research methodology, several references are included here as well.

References for Additional Information on Think Alouds:

Banning, M. (2008b). The think aloud approach as an educational tool to develop and assess clinical reasoning in undergraduate students. Nurse Education Today, 28, (1), p. 8–14. doi: 10.1016/j.nedt.2007.02.001

Bernstein, J. L. (2010). Using “think-alouds” to understand variations in political thinking. Journal of Political Science Education, 6(1), p 49-69. doi:10.1080/15512160903467695

Ericcson, K. A., & Simon, H A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

Forsberg, E., Ziegert, K., Hult, H., & Fors, U. (2013). Clinical reasoning in nursing, a think-aloud study using virtual patients-A base for innovative assessment. Nurse Education Today,

Wainwright, S. F., & McGinnis, P.Q. (2009). Factors that influence the clinical decision-making of rehabilitation professionals in long-term care settings. Journal of Allied Health, 38(3), 143-51.

Wineberg, S. S. (1991) On the reading of historical texts: Notes on the breach between school and academy. American Educational Research Journal, 28(3), 495-519.

For Additional Information on Qualitative Research Methodologies:

Bogdan, R. C., & Biklen, S. K. (2006). Qualitative research for education: An introduction to theories and methods, (5th ed.). Upper Saddle River, NJ: Pearson.

Creswell, J. W. (2002). Research design: Qualitative, quantitative and mixed methods approaches. Thousand Oaks, CA: Sage.

Denzin, N. K., & Lincoln, Y. S. (2012). Collecting and interpreting qualitative materials, Fourth Edition. Thousand Oaks, CA: Sage.



Leave a comment

Descriptive Summary Report of Outcomes of SoTL URG Grants from 2001-2002 to 2014-2015

By Kathleen McKinney, Cross Endowed Chair in SoTL, Illinois State University


For each of the 14 academic years since 2001-2002, there has been a competition for internal Scholarship of Teaching and Learning (SoTL) University Research Grants (URGs).[1] At Illinois State University, SoTL is defined as the systematic study/reflection on the teaching and learning of our ISU students made public. Approximately $20,000 each year was awarded, through a peer review process, for a total of about $280,000 across the 14 years.[2]

Descriptive Data on Recipients/Grants

A total of 63 SoTL URGs were funded in these 14 years. These grants represent work in most departments and in all the academic colleges as summarized below:

  • 6 of 8 departments in the College of Applied Science and Technology
  • 11 of 16 departments in the College of Arts and Sciences
  • 2 of 4 in the College of Business
  • 3 of 3 departments in the College of Education
  • 3 of 3 in the College of Fine Arts
  • 1 of 1 in the College of Nursing

In terms of departments/schools, the most involvement in grants [3] (the total number of times a department/school affiliation was listed by research team members) was from the following departments:

  • Psychology (26)
  • Educational Administration and Foundations (11)
  • Criminal Justice Sciences (10)
  • Communication Science Disorders (9)
  • Teaching and Learning (8)
  • Agriculture (8)

In addition, during two years of the SoTL URG program, applications were required to be from departments, programs, or units and to involve unit teams. Those four grants were awarded to Criminal Justice Sciences, the College of Nursing, the School of Communication, and the Urban Pipeline Program.

Finally, individuals from the following departments/units were also involved in one or more grant teams over the 14 years:

Agriculture, Art, Biology, Family and Consumer Sciences, Dean of Students, English, Health Sciences, History, Information Technology, Languages, Literatures, and Culture, Management and Quantitative Methods, Marketing, Mathematics, Music, Nursing, Politics and Government, Social Work, Sociology/Anthropology, Special Education, Student Government, Teaching and Learning, Technology, and Theater.

Data on Outcomes/Products

Via email requests, lead researchers were asked to submit lists or citations of what they perceived as outcomes/products of their grant(s). Information was obtained for 39 of the 63 grants (or 62%). Therefore, the data reported below is limited to those responses. It is unknown what biases in the data, if any, exist from this response rate.

As the grants were awarded over a period of 14 years, the time frame for the creation of outcomes or products varies from 3 months past the end of the grant/project to 13 years and 3 months past the end of the grant/project. As expected, then, it was often (though not always) the case that there were fewer outcomes from grants just completed in the last couple of years. A condition of the grant funding was that researchers were to present their findings at our local teaching-learning symposium and submit a brief report, reprint, or link to a poster or paper to be posted on line.

Descriptive Data on Outcomes

The outcomes or products submitted for the 39 grants for which responses were received were coded into eight general categories:

  • External publications (e.g., peer reviewed SoTL or disciplinary pedagogical journals) (N=33)
  • Internal publications (e.g., internal SoTL journal, newsletter, department publications etc.) (12)
  • Videos, performances, blogs (2)
  • External presentations (e.g., regional, national, international conferences, workshops, etc.) (50)
  • Internal presentations (e.g., department/campus meetings, our annual teaching-learning symposium) (37)
  • Other grants (internal or external) (5)
  • Under review/in progress (7)
  • Other (positively affected career, used for tenure, used in administrative reports, etc.) (6)

Thus, a total of 152 grant outcomes were listed, which represents an average of just under 3.9 outcomes per reported grant, thus far. There were 47 publications/videos etc. and 87 presentations. The most common outcome submitted was one or more external presentations followed by internal presentations and external publications.

[1] A list of grantees (researchers, departments, and grant title) by year is available at

[2] The total amount was slightly more than this as, in some years, a small additional amount from the Cross Chair in SoTL regular general revenue was added to the URG funds to support part of an additional grant. Four additional grants for a total of $20,000 were awarded for 2015-2016. As these research projects are just beginning and no outcomes have yet occurred, these four grants are not included in this report. This amount and this report also do not reflect the funding and outcomes of SoTL travel grants or mini-grants.

[3] The same individual and his/her unit affiliation were counted each time he/she appeared on a different grant. Some PIs listed student research team members and their departments; others did not. When listed, students’ department affiliations were counted.