The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…

Leave a comment

Does ISU’s Psychology Curriculum Create Science-Savvy Citizens?

Written by Emilio J. C. Lobato, Corinne Zimmerman, and Thomas Critchfield from the Department of Psychology at Illinois State University

Our SoTL project was funded by the office of the Cross Chair in SoTL at ISU to examine ISU students’ involvement in SoTL scholarship or creative work. This topic seemed particularly relevant to our psychology curriculum, as the number of majors pursing out-of-class research experience has increased greatly over the last several years. Currently, of the 5 capstone options for psychology majors, 3 involve hands-on research experience.

Our project investigated whether psychology students who take advantage of research opportunities available at ISU develop a more sophisticated understanding of: a.) science, generally and, b.) psychology as a science, specifically. We examined which “myths of science” students endorse at different levels of the curriculum, and whether endorsement of these myths changes as students matriculate through the psychology curriculum.

To investigate these questions, we administered an online survey to our psychology majors. In order to make sure we sampled students across all four years of our curriculum, we used two recruitment methods. First, we recruited via the psychology department online research participation system (which targets students currently enrolled in courses that offer extra credit for research participation). Second, we used our undergraduate majors listserv (which targets all students, many of whom may not be enrolled in courses offering credit for research participation). We administered this survey twice, once in early fall 2014 and again in late spring 2015.

Across both data collection phases, we administered measures intended to examine student endorsement of myths of science, student perceptions about the nature of science, and student perceptions about psychology as a science. We also asked students about their research experiences with faculty.

Our Findings, So Far

We analyzed the data from fall 2014 and presented our findings at the 2015 University-Wide Teaching & Learning Symposium. Briefly, we found that upperclass students (i.e., juniors and seniors) who had at least one hands-on research experience were more likely to perceive psychology as a science relative to students at any stage of the curriculum without research experience. Additionally, upperclass students with research experience were less likely to endorse myths of science related to whether or not science and scientific methods provide absolute knowledge or proof. A PDF of our poster presentation is available at the symposium website.

Reflections and Issues

Throughout the research process, we encountered several problems. Our goal is to share some of these issues for the benefit of other SoTL researchers. Before we outline these issues, it is important to note that the members of our grant team are all from Psychology, and have decades of cumulative experience working with human participants.

  • Participant Buy-In. Our study was advertised as the “Psychology Majors’ Survey.” Rather than being a standard psychology study, where college students are taken to represent the general population, in our study, participants were told “we are interested in you because you are a psychology major.” Our students signed up to participate, but many failed to take the study seriously. In psychological research, researchers often include items that serve as either manipulation-check questions or attention-check Participants either failed to complete the study or failed attention check questions such as “As an attention check, please select Strongly Disagree for this item.” These questions help us filter out participants who were speeding through the study, answering randomly or choosing the same response for every single question. Ultimately, we ended up collecting 350 response sets across both phases, but only 264 (75%) were usable (142 from fall and 122 from spring). Going forward, researchers using human subjects should be mindful that participants may only be taking part because of the compensation offered, such as extra credit, and fail to take participation seriously. They may answer randomly without paying attention to the questions you are asking or to the instructions you have provided. Without including ways to filter out these kinds of “bad” data, researchers risk drawing incorrect conclusions.
  • Longitudinal Data. Our study design included two phases of data collection, because we were interested in whether the experiences over the course of an academic year resulted in learning, development, or change. An additional problem we encountered was that we only recruited 28 participants who completed the survey at both data collection phases. Given that one of our research aims was to investigate change in the perception of science and endorsement of myths of science, such a small sample size limits the conclusions we can draw. A cross-sectional analysis (such as what we did with our fall data) is a methodologically valid approach to addressing such questions. However, longitudinal research designs that require participants to complete multiple phases of data collection are another compelling source of data. Researchers interested in designing such studies should be aware that participant attrition is a major obstacle. Researchers should design their studies with this in mind and aim to collect far more data at each collection phase in anticipation of participant attrition at subsequent collection phases.

Leave a comment

Beyond the Psychological and Individualistic in SoTL Research

By Kathleen McKinney, Cross Endowed Chair in SoTL, Illinois State University

As a sociological social psychologist, I believe in (from theoretical perspectives) and know (from research data) the importance of demographic/sub-cultural, interpersonal, and situational factors in human behavior. This is not to say that individual traits or states and individual behaviors are irrelevant; they play critical roles as well. But I think we tend to be psychologically focused and individualistic (in the U.S. at least) in our understandings and explanations of behavior and outcomes.

It seems to me that this applies to our views and research on teaching and learning as well. We focus on the role of teacher traits, attitudes, actions or best practices in the classroom. Our addition of an explicit concern with student learning is rather recent and it is my sense that much SoTL work does not systematically include measures of, or theorizing about, student characteristics, interpersonal or relational factors, environmental variables, and institutional characteristics and context.

I realize that this is likely due, in part, to SoTL most often being practitioner research and action research. Thus the perspective on ‘causes’ or correlates of learning is that of the instructor and his/her actions with his/her students. In addition, SoTL is very often classroom or course based. Thus the perspective is on an assignment or technology or intervention by the instructor and its’ role in learning. These perspectives are, in part, what separate SoTL from more traditional educational research. But, I believe SoTL researchers can also take a broader view of what is going on in and around their classrooms, their courses, and with their students in terms of learning.

I do want to acknowledge an exception to the psychological and individualistic emphasis in SoTL research that I have argued exists. It is expected in good SoTL presentations, publications, and other representations of SoTL projects that researchers will describe the context (usually student characteristics and/or institutional variables) of their SoTL studies. This is, in fact, often done by many of us. Such factors, however, are most often shared for comparability, generalizability, and replication rather than as explanatory factors of or correlates to learning.

Thus, I urge you as SoTL researchers to consider and measure –depending on your research question, theoretical perspectives, and your own practitioner knowledge– three sets of factors or variables in your research and their roles in student learning. These are in addition to teacher traits, states, or behaviors and the existence of a course intervention (broadly defined). Taking a look at some of these factors in your SoTL research can give a more complete picture of what is happening and why as well as add to your understanding and knowledge about future application of your SoTL results.

  • demographic/sub-cultural (examples include student gender, age, year in school, social class, race/ethnicity, background knowledge/experiences, goals for the class)
  • interpersonal/relational (examples include amount or type of class interaction, teacher immediacy, level of rapport or conflict among students and between teacher and students, peer relationships or interaction opportunities, student roles in the classroom)
  • situational/environmental/context (examples include number of students, physical aspects of the classroom such as size, technology, seating arrangement, etc., time of the class, purpose or role of the class in the major or the institution, connections to any co-curricular learning opportunities or institutional teaching-learning initiatives)

1 Comment

Common Practical Problems in SoTL Research Design

Written by Jennifer Friberg, SoTL Scholar-Mentor at Illinois State University

Recently, I was reading through a book titled Doing the Scholarship of Teaching and Learning: Measuring Systemic Changes to Teaching and Improvements in Learning (Regan A. R. Gurung and Janie H. Wilson, editors). In this text, Robert Bartsch authored a chapter focused on the practicality of designing SoTL studies and suggested solutions to common practical problems encountered by novice and veteran SoTL researchers. I’ve summarized Bartsch’s advice to SoTL researchers below:

  1. I Have to Measure Everything. It is not unusual for SoTL researchers to be ambitious, designing SoTL studies where masses of data are collected, then analyzed. Practically speaking, there are times when having large quantities of data might not be helpful. Rather, collecting excesses of data can complicate data analysis and yield less useful information than a carefully focused study of  one learning context or stakeholder as a data source. If you need many data points, perhaps consider multiple studies to remain focused on high-quality work. Don’t feel like you need to measure everything at once!
  2. I Do Not Have Many Students. In some disciplines, small class sizes can limit student participant numbers in SoTL research projects which can impact the statistical viability of a SoTL study. Bartsch suggests that selecting the right research design can mitigate issues with sample size. He suggests improving statistical power by using a within participants design such as a pre-test/post-test method. If statistical analysis is difficult secondary to low participant group numbers, consider the use of qualitative measures to study student learning via portfolio/artifact review or student perceptions/reflections. Don’t feel limited by a small participant group!
  3. I Only Have a Single Class. Some SoTL research designs can be used with a single group (or class) of students. However, some researchers might prefer to subdivide a group of students to allow for the comparison of control vs. treatment conditions within a class. Bartsch suggests that there are ways of splitting a single class into multiple groups to address this issue in an ethical manner (e.g., splitting a single class into sections, using different classrooms for each group of students, administration of treatment and control conditions simultaneously within the same classroom context). Use qualitative approaches such as focus groups or interviews to ascertain changes in learning over time. Don’t let a small participant group impact your ability to engage in SoTL!
  4. Random Assignment Sounds Great, But How Can I Do It? Though not all SoTL studies require random participant groups, some will. And, a truly randomized sample is difficult to come by, even if you teach two sections of the same course as there are many situational factors that impact decisions regarding how students register for a class (e.g., day/time of class, students wanting to register with friends, etc.). If you aren’t able to randomize enrollment in class sections or groups in a single class, that is merely a limitation to your SoTL study. Report it, and continue your study. Don’t let a lack of randomization keep you from SoTL research!
  5. I Want to Detect a Subtle Effect. Be realistic in the design of your SoTL study. If you are seeking to research the impact of a teaching support/assignment/approach that is only used once or twice in the semester, it might be difficult to statistically measure learning outcomes. Rather, make a choice to measure larger effects (variables applied often to measure more detectable changes) statistically or choose qualitative methods to gather data to reflect smaller changes in a short amount of time. Don’t miss the opportunity to study incremental student learning!

Are these issues you’ve had as a SoTL researcher? How have you overcome these issues? What was the best approach for you and for your students? We’d love to hear about your experiences in the comments section below.

Blog References:

Gurung, R. A., R. & Wilson, J. H. (Eds.). (2013). Doing the scholarship of teaching and learning: Measuring systematic changes to teaching and improvements in learning. New Directions for Teaching and Learning, Number 136, Winter 2013.

Leave a comment

A Wealth of SoTL Conferences: Making Choices

By Kathleen McKinney, Cross Chair in SoTL, Illinois State University

There are so many conferences related to teaching, learning, and the scholarship of teaching and learning in higher education (HE) that it can be overwhelming to decide which to attend and/or at which to present! There are local, regional, national, and international conferences. Some conferences focus on all levels of education including HE, others on higher education, specifically. Discipline specific conferences and those that are cross- or multi-discipline exist. Some conferences are entirely or primarily SoTL (as defined as evidence-informed study of the teaching and learning of our students and/or work on the field of SoTL). Other conferences are broader including SoTL work, teaching strategies or tips, assessment, policy issues, or other general HE topic areas. Many conferences cover all interesting questions related to teaching and learning; others focus on questions related to only one type or aspect of teaching and learning (e.g., use of technology or civic engagement). Conferences vary in format ranging from those with, primarily, a working institute/workshop format to those with all individual ‘research’ presentations. Most have keynote or invited speakers as well. Some conferences are associated with SoTL professional organizations, others with institutions of higher education, and still others with private profit or not-for-profit organizations.

Below is a somewhat narrow list of multi-discipline SoTL or SoTL/Teaching-Learning in Higher Education conferences for the 2015-2016 academic year. I have attended many of these in the past and always learn something and meet great new people. The culture at these conferences is generally that of support, development, interaction, and generosity. But not all are created equal and which is best for you depends on many factors. As you consider where to attend or present, think about your objectives for and interests in the conference.

  • Do you want to share primarily with people in your discipline or across disciplines?
  • Are you focused on a local audience or want much broader distribution of your work?
  • Do you want to attend, listen, and learn or also make public and teach others about you own SoTL work?
  • How much time, travel funds, and other resources do you have or can you find (check your institution for SoTL or other travel grants in addition to regular travel funding)?
  • What type of conference and conference activities ‘count’ in your institutional reward system?
  • Which conferences might lead to further SoTL networking and professional opportunities? Are there particular people you want to meet or involvement you wish to initiate and where might that best happen?
  • What presentation formats are included in a conference and which best fit as ways for you to represent your SoTL work (e.g., formal paper, panel, poster, video…)?
  • Is the format of the conference and any specific topic focus best for your learning objectives?
  • Are you interested in the keynote speakers and their listed presentations? How will those sessions benefit you and your SoTL work?
  • What other learning opportunities (e.g., workshops) are offered and at what cost?
  • Are you interested in extending your trip or using free time to learn outside the conference (e.g., about the culture, community, etc.) and are there resources or opportunities for that (e.g., tours? transportation?)?

I am sure readers of this blog can think of additional important questions to ask in choosing a SoTL conference. Feel free to respond to this blog post.


October 2–3, Research on Teaching and Learning Summit

Kennesaw State University

Kennesaw, Georgia

October 9–10, Maryville University Scholarship of Teaching and Learning Conference

Maryville University of St. Louis

St. Louis, Missouri

October 15–17, 45th Annual Conference of the International Society for Exploring Teaching and Learning

Savannah, Georgia

October 15–17, Lilly National Conference on College and University Teaching and Learning

Evidence-Based Teaching and Learning

Traverse City, Michigan

October 27–30, 12th Annual International Society for the Scholarship of Teaching and Learning

Leading Learning and the Scholarship of Change

Melbourne, Australia

November 12–14, 2015 Symposium on Scholarship of Teaching and Learning

Connecting People, Practices, and Pedagogies

Mount Royal University

Banff, Alberta, Canada

November 19–22, Lilly International Conference on College and University Teaching and Learning

Evidence-Based Learning and Teaching

Miami University

Oxford, Ohio

January 7–9, Lilly National Conference on College and University Teaching and Learning

Evidence-Based Teaching and Learning

Austin, Texas

February 10–12, 8th Annual Conference on Higher Education Pedagogy

Virginia Tech

Blacksburg, Virginia

February 18–20, Lilly National Conference on College and University Teaching and Learning

Evidence-Based Teaching and Learning

Newport Beach, California

March 31–April 1, The SoTL Commons Conference

Savannah, Georgia

June 3–5, The Teaching Professor Conference

Washington D.C.

June 21–24, Society for Teaching and Learning in Higher Education

Empowering Learners, Effecting Change

London, Ontario, Canada

Leave a comment

Opportunities to Publicize and Share your SoTL Work Locally

From the Office of the Cross Chair in SoTL, Illinois State University

If you have completed SoTL projects and/or or representations of the project and results (publication citations, web urls…), please consider sending or submitting the appropriate materials to publicize and share your SoTL work locally. Below are some opportunities to do so (and the links to additional information).

Gauisus is the ISU ‘published’, peer-reviewed, on-line, multi-media SoTL publication. SoTL on the teaching and learning of Illinois State University students represented as research papers, notes, essays, posters, websites, videos, etc. are all considered for publication. Currently, Gauisus is published each spring with submissions due in late fall. Details on submitting your work are available at

The Office of the Cross Chair is seeking submissions (500-750 words) for publication on ISU’s SoTL blog: The SoTL Advocate. Blog posts can describe faculty/student SoTL projects, review or summarize recent SoTL research articles, discuss advances in disciplinary SoTL, etc. Contact Jen Friberg ( for more information!

The SoTL at ISU Newsletter is published every September and January. We welcome 250-500 word summaries of your SoTL work about the teaching and learning of Illinois State University students for inclusion in the newsletter. Summaries should include the purpose, method, results, implications, and—if appropriate—applications made and relevance to other disciplines of the SoTL project. Submissions of such summaries are due August 1 or December 1 as a word file to In your email, please indicate the summary is for the newsletter.

SoTL researchers at ISU are profiled in the ‘highlight box’ on our SoTL home page. Send your name, department, photo, and a 2-3 sentence summary of your SoTL work (with a url link if you wish) to for consideration. In your email, please indicate the material is for a possible webpage profile.

If you have made your past or current SoTL work public beyond the local level (through published work or a video or a web site…), please share citations to that work ( These will be added to the list of SoTL publications by Illinois State University faculty, staff, or students on the SoTL website.

Every January a full day symposium on teaching and learning at Illinois State University is organized by the ISU Center for Teaching, Learning, and Technology. Papers and posters on scholarly teaching and SoTL are presented. Lunch and a keynote presentation are also part of the event. Please submit your SoTL work to the symposium to share with your ISU colleagues. (