The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Student Evaluations as a Starting Point for SoTL

Written by: Jennifer Friberg, SoTL Scholar-Mentor at Illinois State University

Faculty members across the country have begun the first weeks of a new academic semester. And, while faculty are focused on current students, feedback from fall semester student evaluations will soon be available (and perhaps already have been shared) for faculty to review and ponder. This begs the question, how can faculty use student evaluation data effectively when they’re already engaged with a new set of students? What is the best way to consider former students’ perspectives to improve current and future students’ experiences?

A recent post from “TILT,” the University of Minnesota’s Center for Educational Innovation weblog, outlines important considerations for faculty in interpreting student evaluation data, suggesting resources that could be of help to faculty in making sense of their students’ perceptions. They also mention that faculty should take student evaluation data seriously, as:

 …in general, student ratings tend to be statistically reliable, valid, and relatively free from bias or the need for control, perhaps more so than any other data used for faculty evaluation (TILT, 2014).

Additionally, the aforementioned TILT blog post provides two suggestions for faculty engaged in review of student evaluation data: meeting with someone to talk about their student feedback and gathering midterm feedback. These are both excellent ideas, as both have the ability to lead to dialogue, understanding, and change in both teaching and learning. On Illinois State University’s campus, resources for teaching-specific support (including help with “midterm chats”) can be found through the Center for Teaching, Learning, and Technology.

Beyond these suggestions, it is quite possible that student evaluation data can act as a fertile foundation for SoTL research. By definition, SoTL research is intended to provide a systematic reflection on teaching and learning. As students’ reflections of their own experiences are inherent in end-of-semester student evaluations, feedback provided regarding course objectives, syllabus construction, specific assignments/projects, tests/quizzes, and various modes instruction might raise questions about their effectiveness in driving student learning. As faculty, we cannot assume that our students are learning because we “think” that is the case.. Rather, we must have evidence to prove it! Thus, student evaluation data can lead us to design SoTL projects to obtain such evidence.

Faculty looking to use student evaluation data as a starting point for SoTL research, should consider the following SoTL research approaches:

  • In-Course SoTL: Faculty can use student evaluation responses to inspire an in-course SoTL research project answering questions about a specific course they teach. Faculty can structure a SoTL study to investigate the effectiveness of particular pedagogical approaches (e.g., projects, assessments, reflection, small group discussions, online instruction) on student learning. While results from in-course SoTL studies may lack generalizability to other cohorts of students or to other disciplines, they are useful in making changes in a course to increase student learning for future students enrolled in the course being studied.
  • Intra-Disciplinary SoTL: Faculty can collaborate with colleagues from within their discipline to design a SoTL study to answer teaching and learning questions that become evident on student evaluations and can be applicable across course boundaries. For instance, faculty in nursing or dietetics might investigate how clinical personas develop in undergraduate students during their first practical internships. Sociology professors might investigate the evolution of the sociological imagination across courses within the major. Results from SoTL studies of this nature are considered to be more generalizable, and can impact student learning across a particular program or discipline.
  • Inter-Disciplinary SoTL: Faculty who seek to engage in inter-disciplinary SoTL research involve faculty from other disciplines (and possibly other universities) to study an aspect of teaching and learning. Faculty might share similarities yielded from student evaluations and find that multiple disciplines are seeking to better understand, for example, how technology impacts student note taking practices, or how social media impacts student perceptions of disability.

At Illinois State, the office of the Cross Chair in SoTL provides many resources to help assist faculty in developing their own SoTL projects derived from student evaluations or any other source. Please consider contacting the Cross Chair (sotl@ilstu.edu), attending the upcoming SoTL Brainstorming workshop, or applying for a small SoTL grant to support planned SoTL work.

References:

TILT. (2014, Dec 15). Reviewing student evaluations of teaching… [Web log comment]. Retrieved from https://uminntilt.wordpress.com/2014/12/15/reviewin-student-evaluations-of-teaching/


1 Comment

Developing SoTL Research Ideas and Questions

Written by Kathleen McKinney. Cross Endowed Chair in SoTL at Illinois State University

As we plan for a small, informal SoTL Idea/Project Brainstorming Session here at Illinois State University on February, I am thinking, again, about the processes of developing ideas, research questions, hypotheses, or problems for SoTL projects.

Where do we, can we, find ideas for the research foci of our SoTL work? Let me suggest six sources of ideas on which you might draw. Of course, this is not an exhaustive list and although I list these separately, they should be integrated as you use multiple of these sources to draft, refine, improve, and finalize your SoTL research project idea.

  1. First, and perhaps most importantly given that SoTL is action, practioner research, we garner ideas for our SoTL projects from our lived experiences with students and teaching both in and outside the classroom. Often a SoTL project arises because we notice changes in student attitudes or behaviors as a result of something we did in the classroom and we want to validate that or we are concerned about something that does not seem to be going quite right in our class or program and we want data on why or data on what might ‘fix’ it, or we believe there might be interesting learning as a result of a co-curricular experience that we want to confirm and strengthen or… well, you get the idea!
  2. Second, students can and should be a source of ideas for our SoTL research. Students can implicitly and explicitly offer SoTL project ideas via informal conversations with us, data from student evaluations or classroom assessment, interactions in our student disciplinary clubs, and as our co-researchers on SoTL projects.
  3. Third, as with all research, we should be looking to fill the gaps in the existing SoTL literature. As I have noted in an earlier post, some gaps are lack of SoTL on graduate student learning, lack of longitudinal studies, lack of projects at levels beyond or outside the classroom, lack of studies that directly measure intervening variables and processes (the why and how), and a lack of projects that build praxis—SoTL study followed by application of those results in context then a follow-up study, more application, more follow-up research… And, of course, our SoTL work should build on prior research.
  4. Fourth, we should all (especially those of us not in an Education discipline) be learning about models and theories of learning in higher education. Such theories can certainly suggest ideas for our SoTL research.
  5. Fifth, let’s not underestimate the benefits of colleagues as ‘sounding boards’ for our ideas. Listen to and talk with colleagues from your discipline and other disciplines at, for example, teaching improvement or SoTL campus events, at disciplinary or SoTL conferences, and in department or committee meetings.
  6. Sixth, consider institutional/association priorities on your campus or in your discipline related to teaching and learning. What are the goals and objectives in your institution’s strategic plan that relate to learning in your classes? What are the best practices recommended for teaching by your disciplinary society? How do these inform your ideas for a SoTL project?

Below are just a few references related to the topic of this post:

  • Bass, R. (1999). The Scholarship of Teaching: What’s the Problem? Inventio: Creative Thinking    about Learning and Teaching. 1(1), 1-10.
  • Hutchings, P. (2000). Opening Lines: Approaches to the Scholarship of Teaching and Learning.  The Carnegie Foundation for the Advancement of Teaching.
  • McKinney, K. (2007). How Do I Get Started? Chapter 3 in Enhancing Learning through the      Scholarship of Teaching and Learning: The Challenges and Joys of Juggling. Anker/Jossey-Bass.
  • Nelson, C. (2003). Doing it: Examples of Several of the Different Genres of the Scholarship of     Teaching and Learning. Journal on Excellence in College Teaching. 14(2), 85-94.


Leave a comment

SoTL Funding Opportunities for Illinois State Faculty & Staff

ISU Faculty and staff members should be on the lookout for information (in campus mail, on isuteach, on www.sotl.ilstu.edu, from you Chair/Director…) on several types of support/development for your scholarship of teaching and learning (SoTL) research and funding opportunities related to your SoTL work over the next several weeks. Only ISU faculty and staff will be eligible for these funding opportunities. Questions can be emailed to sotl@ilstu.edu. These opportunities will include the following:

A Variety of SoTL Support/Development Opportunities

This spring, we plan to offer a workshop on SoTL and IRB and another on Applying SoTL Results. We will also likely have a SoTL Reading Group. We are happy to read and provide feedback on drafts of your SoTL products. We welcome submissions to our SoTL at ISU newsletter, guest submissions to this blog, and submissions to our 3rd issue of Gauisus next fall (http://gauisus.weebly.com/). We are considering a summer institute on designing a SoTL project. We hope to have additional travel funds next fiscal year. Information related to these opportunities will be shared over the next several weeks.

The Walk the Talk Scholarship of Teaching and Learning Contest for the best example of application of SoTL knowledge beyond the individual classroom

The Cross Endowed Chair in the Scholarship of Teaching and Learning will be seeking applicants for the 2015 Walk the Talk contest. This contest is intended to recognize the best team or academic unit who applied own or others’ SoTL research beyond the individual classroom to solve a problem, achieve a goal, or exploit an opportunity resulting in improved teaching or enhanced student learning. The winning team or unit will receive $2,000 in operating funds. Applications will be due mid-March 2015.

Scholarship of Teaching and Learning Scholar-Mentors: Call for Applications for FY16

For the 2015-2016 academic year support for the SoTL on our campus will come from a part-time, Interim Cross Chair in SoTL and up to two Illinois State University SoTL Scholar-Mentors per semester. As a SoTL Scholar-Mentor, you will have reassigned time to the Office of the Cross Chair in SoTL to work on a SoTL project of your own and, primarily, to engage in tasks that support the SoTL work of colleagues at ISU and beyond. All tenured or tenure-track Illinois State University faculty members with experience in SoTL are eligible. Successful applicants will be granted reassignment of one course for fall 2015 and/or spring 2016 semesters. Home departments/schools will receive a 25 percent buyout of faculty salary per semester for any faculty member serving as a SoTL Scholar-Mentor. SoTL Scholar-Mentors will receive the course reassignment and $3,000 in operating funds. Applications will be due late March 2015.

Scholarship of Teaching and Learning Small Grant Program 2015-2016

The Cross Endowed Chair in tSoTL requests proposals for the SoTL Small Grant Program. The program provides SoTL small grants to study the developmental and learning outcomes of ISU students. At ISU, through our work with the Carnegie Academy for the Scholarship of Teaching and Learning (CASTL) Campus Program, we define SoTL as the “systematic study/reflection on teaching and learning [of our ISU students] made public.” This definition allows for research in any discipline and the use of various methodologies. The work may be quantitative or qualitative in nature and focus on class, course, program, department, cross-department, and co-curricular levels. For 2015-2016, projects must focus on a teaching-learning issue(s) explicitly related to one or more of the goals in the University’s strategic plan, Educating Illinois. Grants of up to $5,000 are available. We expect to award 4-5 grants. Proposals will be due mid-May 2015.

The Dr. John Chizmar & Dr. Anthony Ostrosky Scholarship of Teaching and Learning Award Call for Applications- 2015-2016

This University-wide award is designed to recognize and encourage high quality and quantity of SoTL work at ISU and in the discipline beyond campus that contributes to the SoTL field, the SoTL body of knowledge, improved teaching, and enhanced student learning. All faculty members and academic staff at ISU are eligible. The recipient is recognized at Founders’ Day, receives $3,000 in funds, and is asked to give a talk about his/her SoTL work. Portfolios will be due September 30, 2015.


2 Comments

Tips for Publishing SoTL Work

Written by Kathleen McKinney, Cross Endowed Chair in SoTL at Illinois State University

There are many ways to represent and share SoTL work, one of which is the traditional, academic journal article. As a past pedagogical journal editor and member of many editorial teams, I suggest the following tips re publishing SoTL work in this form.

1. Don’t assume you have invented the wheel to the SoTL vehicle. There is very likely relevant prior literature on same or related SoTL question; in your and/or related disciplines. You need a useful, relevant, coherent literature review. We often see papers with missing literature reviews or literature reviews about the research on the disciplinary topic not the SoTL project.

2. Make clear, and do so early on in the paper, what your SoTL questions and research purposes are, and how they fill a gap or build on past work (don’t just reinvent the wheel). And then stick to them. We often see the questions or purpose shift or change in later parts of the paper.

3. Don’t be atheoretical. Too much SoTL work is atheoretical. Are concepts or models from your discipline relevant to the SoTL questions and results? What about concepts, models, theories from other fields and higher education in general? Theory can be deductive or inductive. Theory can be in the literature/introduction and/or in the discussion.

4. Include information about ‘context.’ By definition, SoTL work is local, context-specific, action research. Thus, readers need to know your context to understand, evaluate, and use your work. Briefly give some information about the institution, department or discipline, class and/or students… Don’t make readers guess your context or assume it is similar to theirs.

5. Provide sufficient detail in methods and measures (and appendices) so readers can understand and replicate or adapt (or question or criticize). Such detail is often missing in SoTL papers.

6. Have actual SoTL data (especially on learning; maybe multi-method/measure). Do not submit s “I tried it, I liked it” or “I tried it, my students liked it” papers with no or anecdotal data.

7. SoTL is action, practioner, applied research. Spend time in the discussion section talking about how you have used the results and/or plan to use the results (specific changes and actions) to enhance student learning. Make application suggestions for readers. You would be amazed at how often this area is neglected in SoTL papers.

8. Remember you may have a multi-discipline audience for your SoTL work. Watch use of jargon. Draw on literature from other disciplines. Help reader learn from your results. Discuss applications generally. Suggest adaptations to other disciplines.

9. Be persistent. There are many good SoTL journals in which to publish your work.

10. And, of course, do all the important things you would do submitting any paper for review:

  • Know the mission/type of work published in various outlets; select one the paper fits.
  • Contact the editor to see whether paper is a fit to the journal you are considering.
  • Don’t miss citing relevant literature published in the journal to which you are submitting.
  • Let others read and comment then rewrite before you send paper to journal reviewers.
  • Give appropriate credit to work you build on, measures you borrow, or theory you use.
  • Carefully edit and proof read.
  • Take reviewer suggestions seriously; do all that make good sense and are doable.
  • When you resubmit, include a letter to the editor and reviewers outlining all the changes you made and noting any they suggested that you did not make and why you did not.