The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Thoughts on SoTL Advocacy from the SoTL Commons Conference

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences & Disorders at Illinois State University (jfribe@ilstu.edu)

A few weeks ago, I had the good fortune of being invited to deliver one of two keynote addresses at the annual SoTL Commons conference in Savannah, Georgia. Happily, I was given the opportunity to select my own topic for my talk and, having thought deeply about several options, selected SoTL advocacy as my focus. This is likely not a surprise to those who know me, as I am a passionate advocate for research on teaching and learning. After developing several iterations of my talk, I chose to focus my remarks on five ideas I believe to be central to effective SoTL advocacy. I share them here, in the hopes that one or more of these might resonate with folks for use now or at a later time in their own SoTL advocacy efforts.

As a starting point, I do feel as though the above screenshot of one of the slides from my keynote hits on something very important: SoTL advocacy should be undertaken in ways that employ diverse approaches to our advocacy work. Perhaps the the word “customized” might even be appropriate as a corollary to this recommended diverse approach to advocacy, as efforts to engage an expanded group of stakeholders in SoTL should be specifically tailored to fit the contexts in which SoTL advocacy is being undertaken. With that in mind, suggestions for thoughtful and purposeful SoTL advocacy presented at the SoTL Commons included the following:

  1. Keep your SoTL “start-up” story in mind. Share it with others, as understanding your interest in SoTL might drive someone else to develop an interest, too. I have found this to be true, particularly for colleagues within your own discipline. My field of speech-language pathology has an established standard for using evidence-based practices to inform clinical decision-making. When I explain to other speech-language pathologists or audiologists that I started with SoTL because of my view that evidence to support my teaching practices is just as necessary as evidence to support my clinical work, folks can easily understand my perspective. While they might not engage in SoTL, they can conceive of how it might be important to others and to the discipline, at large.
  2. Develop an “advocative” (ad-VOCK-ah-tiv) mindset. Encourage people to think about SoTL in different ways, via a lens of provocative advocacy. The central idea to being advocative is being both thoughtful and purposeful in advancing (in this case) SoTL. Think about why advocacy is needed with a person or group. Plan a thoughtful approach to your advocacy efforts, one that makes the stakeholders you seek to engage leave their interaction(s) with you changed in their thinking about SoTL. If you find yourself having similar conversations across a variety of stakeholders, that’s okay, as being advocative can be necessarily repetitive!
  3. Consider the advantages of code switching. I have facilitated a particular undergraduate language development course over a dozen times in the last decade at my university. One of the important concepts in that course’s curriculum is that of code switching, the notion that children learn to adjust the language they use (tone, vocabulary, delivery) based on who they are communicating with. I would argue that advocacy efforts require a similar type of code switching to make SoTL matter to a given audience. As there are very different stakeholder groups for SoTL (e.g., faculty, students, administration, accreditation groups), it is important to speak to language of the individuals you seek to engage in your advocacy efforts. SoTL should be made important to individual stakeholders in individual ways.
  4. Establish semantic congruency with specificity. We often lack semantic congruency in our discussions about SoTL. Why? A variety of words and phrases are used to talk about research on teaching and learning, which can lead to confusion (as discussed in this blog post a few weeks ago!). If you’re talking with folks about SoTL, be able to identify similarities and differences between SoTL and educational research, action research, or classroom-based research. Develop ways to describe well that which you advocate for.
  5. Mentorship is a critical component of SoTL advocacy. With experience, many SoTL scholars become mentors to novice student or novice/veteran faculty SoTLists. While this is wonderful, I would argue that mentees need to observe not only the work that goes into a SoTL project, but advocacy efforts to advance that work. This type of mentorship includes the sharing of practices and processes for self-advocacy and collective advocacy at any point in a project’s lifespan (pre, during, post) to advance SoTL at micro through mega levels of impact.
Advertisements


Leave a comment

The Mind of SoTL

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences & Disorders at Illinois State University

Today’s blog is a little different than most I write, but is offered as a reflection on an important part of SoTL for many of us: the people we are fortunate to work with to better understand the dynamic duo of teaching and learning. My thoughts here were inspired by a mid-morning video conference call today with two of my favorite people in the world: Sarah Ginsberg and Colleen Visconti, both SoTL enthusiasts and fellow professors of speech-language pathology. I met them almost a decade ago when we all served on a coordinating committee for a special interest group in the American Speech-Language-Hearing Association. Our work on that committee led to our first professional collaboration, as well as cherished, enduring friendships.

Today’s call focused on our next project together, an edited book focused on the application of research on teaching and learning in the clinically-based classroom (stay tuned for our call for submissions!). I knew before the call started that our conversation would be thoughtful, collegial, fun, and productive. I was not wrong. The thing is, though, most of my interactions with folks around various SoTL topics make me equally happy, both personally and professionally. Conversations with others have confirmed that I am not alone in this! One has to wonder why…

Since becoming involved with SoTL almost a decade ago, I have found it remarkable that cross-disciplinary groups of higher ed stakeholders can occupy the space surrounding SoTL — almost uniformly — with such positive intentionality. We are diverse in discipline, culture, language, thought, and praxis, but we are united by our passion for teaching and learning. I would offer that there’s something unique about this shared focus that transcends anything other than a true desire to advance our SoTL discipline. In that vein, we are invested both cognitively and emotionally in our SoTL work.

A few years ago, I wrote a blog post that offered: while the heart of SoTL is in the classroom* — and likely always will be — it was made clear to me last week that the mind of SoTL is focused on interactions and relationships that advance our knowledge of teaching and learning. I think that this notion of the “mind of SoTL” being focused on interactions and relationships is more crystallized for me now than it was two years ago. I’ve come to understand that even my solo SoTL work isn’t truly solo. It focuses on the intricacies of the teacher/learner dynamic in an effort to change future interactions for the better. Through my collaborative SoTL work, I’ve developed a wide network of fellow SoTLists who challenge and inspire me, and, through my interactions with them, bring joy to the work that I already love to do. Truly, I am thankful every single day to be a part of the global SoTL community.


Leave a comment

SoTL, ER, and DBER: Thoughts Inspired by a Twitter Conversation

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences and Disorders at Illinois State University

A few weeks ago, I published a blog post titled “Finding the Goldilocks fit for your SoTL manuscript.” As always, I publicized the new blog post on my Office of the Cross Chair Twitter account (@ISU_SoTL). Who knew that a really great question from Erin Whitteck (@EWhitteck) would engender such a great conversation over the following days?

tweetstorm

Folks contributing to the subsequent tweet stream offered the suggestion that there is overlap between disciplinary-based educational research (DBER), SoTL, and educational research (ER), but that the lines between these types of inquiry could be a bit blurry. Questions were raised about rigor, methodological differences, and resources for better understanding. Since then, I’ve been pondering. To get us into the same semantic sandbox, consider the following definitions:

SoTL “involves the systematic study of teaching and/or learning and the public sharing and review of such work. ‘Study’ is broadly defined given disciplinary differences in epistemology and the need for interdisciplinary SoTL…SoTL focuses on teaching and learning at the college level, and is primarily classroom based. Ideally, SoTL also involves application and use” (McKinney, 2007, p. 10).

“ER is the scientific field of study that examines education and learning processes and the human attributes, interactions, organizations, and institutions that shape educational outcomes. Scholarship in the field seeks to describe, understand, and explain how learning takes place and how formal and informal contexts of education affect all forms of learning. Educational research embraces the full spectrum of rigorous methods appropriate to the questions being asked and also drives the development of new tools and methods” (AERA, 2018).

“DBER is grounded in the science and engineering disciplines and addresses questions of teaching and learning within those disciplines…DBER investigates teaching and learning in a discipline using a range of methods with deep grounding in the discipline’s priorities, worldview, knowledge, and practices…DBER is informed by and complementary to general [educational] research on human learning and cognition” (Singer, Neilsen, & Schweingruber, 2012, p. 9).

In response to the suggestion that there is overlap between SoTL, ER, and DBER, I believe that to be an undeniable truth. Each focuses on research on teaching and learning, serves to add knowledge to better understand educational processes, demands rigor, and has the potential for impact across contexts (e.g., micro, meso, macro, mega). SoTL, DBER, and ER also each purport to embrace a wide array of research approaches, including qualitative, quantitative, and mixed methods designs.

So, then, what about the differences? Here are a few that are important to consider:

  1. Both DBER and ER apply to K-12 research in addition to research in higher education. SoTL is focused on higher education.
  2. DBER is a form of ER, with a focus on science and engineering disciplines.
  3. The purpose of SoTL is to advance the practices of teaching and learning through systematic study and reflection (Larsson, Martensson, Price, & Roxa, 2017). The purpose of ER and DBER is to create generalizable knowledge about teaching and learning (Felten, 2015), though it should be noted that DBER scholars acknowledge a need to translate DBER findings to classroom practice, in line with SoTL (Singer, Neilsen, & Schweingruber, 2012).
  4. A common tenant of SoTL is that scholars study their unique learning contexts to better understand their teaching and/or their students’ learning. Most ER removes the investigator from the context being studied.
  5. While ER and DBER seek to create generalizable findings, most SoTL is not inherently generalizable as it often studies a single learning context and might study a small(ish) number of individuals. Rather, SoTL should be inherently replicable through the explanation of a systematic approach to investigation that is reported when results are disseminated. SoTL seeks to build generalizability over time as different constructs are studied in different places by different people at different times.
  6. SoTL embraces a “big tent” philosophy with a wide array of disciplines and diverse approaches to inquiry recognized as making important contributions to research on teaching and learning. As ER and DBER typically focus on education or STEM fields, theories, methods, and practices for these disciplines are typically utilized in those types of inquiry.

ER, DBER, and SoTL are all valuable forms of teaching and learning research. While there is overlap between and across these categories of research, they are not competitors. They exist on a continuum that encourages scholarly approaches teaching and further research on teaching and learning. I would argue that it is the interpretation of the similarities and differences of SoTL, ER, and DBER that friction might emerge, as we typically consider research through our own disciplinary lenses. That might be topic for a future blog all on its own…

So, Erin, I’ll try to answer Twitter question from earlier in November that launched this discussion: “what is the difference between a disciplinary SoTL journal and a DBER journal?” Honestly, there may not be a difference. In some fields, SoTL and DBER might both be published in the same journal. In others, it might be one or the other. I’d suggest that you look at the aims and scope statements for your discipline’s SoTL and DBER journals. Identify which aligns with the work you’ve done in terms of purpose (e.g., add or apply knowledge). If you’re not sure, editors LOVE getting emails from prospective contributors. I really mean this! Send an abstract of your work and ask if it’s suitable for their journal or ask a question or two to guide your efforts. Good luck!

Blog References:

American Educational Research Association. (2018). What is educational research? Downloaded from http://www.aera.net/About-AERA/What-is-Education-Research.

Felten, P. (2015). Principles of good practice in SoTL. Teaching & Learning Inquiry, 1(1), pp. 121-125.

Larsson, M., Martensson, K., Priace, L. & Roxa, T. (2017). Constructive friction? Exploring patters between educational research and the scholarship of teaching and learning. Paper presented at the 2nd EuroSoTL Conference, Lund, Sweden.

McKinney, K. (2007). Enhancing learning through the scholarship of teaching and learning: The challenges and joys of juggling. Anker Publishing: Boston, MA.

Singer, S. R., Nielsen, N. R., & Schweingruber, H. A. (Eds.). (2012). Discipline-based education research: understanding and improving learning in undergraduate science and engineering. National Academies Press: Washington, D.C.


2 Comments

Finding the Goldilocks fit for your SoTL manuscript: It’s a question of content, voice, and application!

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences and Disorders at Illinois State University (contact email: jfribe@ilstu.edu)

As is the case with disciplinary research, SoTL research is carried out carefully and systematically. Data is analyzed, results are presented, and a compelling case is made for the implications of the outcomes of SoTL research process. For those of us for whom a peer-reviewed journal article is the “currency” of academic productivity, we think about where we might eventually send our work for review and (hopefully!) publication throughout our project’s life. We search lists of SoTL publication outlets seeking the Goldilocks “fit” for our research, carefully reviewing the aims, scopes, and missions of SoTL journals as part of this process. As these efforts unfold, there is a foundational question that must be asked as part of the search for a journal “home” for your SoTL work: Does my SoTL best fit in a disciplinary journal or a cross-disciplinary journal?

To make sure we are all on the same page semantically, I’d define a disciplinary SoTL journal as one that focuses primarily on one discipline. Teaching and Learning in Communication Sciences and Disorders is one that is a great example of this, with its focus on SoTL for the connected disciplines of speech-language pathology and audiology. Teaching and Learning Inquiry would be an ideal example of a cross-disciplinary SoTL journal, as manuscripts selected for publication potentially apply to a variety of disciplines across the academic spectrum.

The question of disciplinary versus cross-disciplinary fit has to do (mainly) with the potential reach for your work. For instance, if you conduct a rigorous SoTL project to understand how art history students’ learning is impacted through study abroad experiences in Italian museums, it’s possible that your findings might have primary interest and impact within the discipline of art history. As such, a journal like Art History Pedagogy & Practice would be a wonderful outlet for your work. A study on intrapersonal learning as a result of students’ involvement with an array of campus student organizations might have a broader disciplinary appeal, with publication in the cross-disciplinary Journal of the Scholarship of Teaching and Learning one potential outcome.

That said, it’s often how a manuscript is written that truly influences its fit for publication. With this in mind, three primary considerations become evident:

  • The content of your manuscript is extremely important. Is the topic being explored centered on questions from a single discipline? Or, might the content of your paper be of interest to people representing a variety of disciplines and contexts?
  • Your writing voice is also critical. When you constructed your manuscript, did you use accessible terminology or did you employ disciplinary jargon to best make your points?
  • How have you described the potential applications of your work? Did you tie your findings to uses and impacts in one discipline or did you make an effort to extend your research outcomes to a variety of fields and contexts?

The decision tree below operationalizes the notions of content, voice, and application through the lens that the more linguistically accessible and contextually inclusive your manuscript seeks to be, the more likely it is to find a fit in a cross-disciplinary SoTL journal.

SoTL decision tree

I have one last thought for your consideration. Some SoTL is simply so focused on one discipline that its contributions to the pedagogical content knowledge of that discipline must be honored with publication in a disciplinary journal. Similarly, some SoTL cannot be tied to only one discipline, or perhaps it’s so applicable to other disciplines that publishing in a cross-disciplinary outlet is its best fit. Thus, SoTL is not “better” or “worse” if its published in a disciplinary rather than a cross-disciplinary journal — or vice versa. Rather, it’s knowing where your SoTL belongs that helps it to have value to your audience. 

 


Leave a comment

The SLaM Model of Applying SoTL In and Beyond One Classroom

Written by Kathleen McKinney, Illinois State University, Emeritus and Jennifer Friberg, Illinois State University 

slamIn this blog post we share a model for the application of scholarship of teaching and learning findings in and beyond the individual classroom level. The model, named SLaM, is detailed in the Introduction chapter of our edited book, Applying the Scholarship of Teaching and Learning Beyond the Individual Classroom (Indiana University Press, 2019, in press). The focus of that volume is on SoTL and its application beyond one classroom but the SLaM model is about application at any level. We define SoTL using both our institutional definition, ‘the systematic reflection/study of teaching and learning made public,’ as well as key characteristics as practitioner, action reflection/research that is usually about the instructor/researchers’ own students and/or students in their discipline and is most often at the local level. We understand application as the use of SoTL research findings and implications to design, change, intervene, make decisions, etc., primarily in institutions and disciplines, to enhance teaching and student learning.

The SLaM model is an outgrowth of our early discussions of application at various levels (e.g., Friberg & McKinney, 2015, 2016; McKinney 2003, 2007, 2012).[1] We then organized and built on those ideas, as we wrote for and edited our latest book, to create the SLaM model. The model uses three questions to conceptualize, categorize, and understand the use of SoTL results/knowledge in applications to teaching and learning. We briefly note these here but a more detailed discussion, diagram, and examples of the model can be found in our Introduction to our edited book (see endnote 1 below for the citation for the model).

  1. What is the source of the SoTL that is applied? The “S” in our SLaM framework is connected to identifying the source(s) of SoTL findings being applied. SoTL research results that are applied at various levels may be from the teacher’s original scholarship of teaching and learning studies, SoTL work by colleagues, the synthesis of presented or published SoTL research in the discipline/institution/larger SoTL field, or some combination of these sources of SoTL results and implications.
  2. At what level(s) are the data/results/implications applied? There are numerous levels (the “L” in our framework) at which SoTL findings and implications could be applied to positively impact teaching and learning. These levels include the individual classroom, course/module, program, department, college, co-curricular, institutional, disciplinary, multi-institutional, and multi-disciplinary levels.
  3. What mechanisms or processes are used (or could be used) to apply the SoTL data/results/implications to new areas or contexts at various levels? The “M” in our SLaM framework represents the many mechanisms that exist or could be created that can be used as processes for novel applications of SoTL findings. A few examples include assessment, quality assurance, course/program design or redesign, accreditation, budget development, strategic planning, faculty/staff development, interdisciplinary initiatives, and graduate student training.

In our forthcoming edited book, eleven examples of the application of SoTL are described; two in our Introduction and nine in the contributed chapters. We briefly summarize three of these examples of applications and their fit with our model here. First, Brent Oliver, Darlene Chalmers, and Mary Goitom of Mount Royal University in Canada in their chapter, “Reflexivity in the Field: Applying Lessons Learned from a Collaborative Scholarship of Teaching and Learning Study Exploring the Use of Reflexive Photography in Field Education” use findings and implications from face-to face interviews with students from multiple institutions (source). They apply what they learned at the course, program and department levels using curricular reform, program review and accreditation (mechanisms). They are planning additional applications in a new interdisciplinary fellowship program and via faculty development programs.

Another example comes from Belgium. In the chapter, “Feedback First Year”- A Critical Review of the Strengths and Shortcomings of a Collective Pedagogical Project,” Dominique Verpoorten, Laurent Leduc, Audrey Mohr, Eléonore Marichal, Dominique Duchâteau, and Pascal Detroz describe their sources of SoTL findings: SoTL literature on feedback practices as well as original data from interviews with members of the faculty participating in SoTL staff development programs, observations and diaries of advisers, minutes of meetings, and descriptive templates of project outcomes. Levels of application included individual courses, faculties/departments (group of courses; program), and institution. The mechanisms they used for application were specific course re-design tasks (designing feedback activities by faculty participants), a variety of course interventions, and sharing results in departments via meetings and plenaries.

Finally, contributors Claire Vallotton, Gina A. Cook, Rachel Chazan-Cohen, Kalli B. Decker, Nicole Gardner-Neblett, Christine Lippard, and Tamesha Harewood share their SoTL applications in “The Collaborative for Understanding the Pedagogy of Infant/toddler Development: A Cross-University, Interdisciplinary Effort to Transform a Field through SoTL.” Their project used implications from past SoTL literature, reflection, and original SoTL studies on multiple campuses (sources) at the course, program, department and disciplinary levels. The application mechanism was a cross-institutional, collaborative group of scholars (CUPID) where participants shared resources, conducted research, and disseminated work via conferences, workshops, publications, meetings.

We hope readers of this blog post will take a look at the details of the SLaM model and the interesting projects and applications from around the globe presented in the edited volume. We welcome feedback on the model and hope others will find it useful in their SoTL research and applications.

Blog References

Friberg, Jennifer C., and Kathleen McKinney. 2016. “Creating Opportunities for Institutional and Disciplinary SoTL Advocacy and Growth.” Presentation. SoTL Commons Conference, Savannah, GA, USA.

Friberg, Jennifer C., and Kathleen McKinney. 2015. “Strengthening SoTL at the Institutional and Disciplinary Levels.” Poster presentation. EuroSoTL, Cork, Ireland.

McKinney, Kathleen. 2012. “Making a Difference: Applying SoTL to Enhance Learning.” The Journal of the Scholarship of Teaching and Learning 12(1): 1-7.

McKinney, Kathleen. 2007. Enhancing Learning through the Scholarship of Teaching and Learning: The Challenges and Joys of Juggling. San Francisco: Jossey-Bass.

McKinney, Kathleen. 2003. “Applying the Scholarship of Teaching and Learning: How Can We Do Better?” The Teaching Professor August-September:1,5,8.

 

[1] As discussed in the Introduction to our edited book, the SlaM model overlaps slightly with the 4M model (Poole and Simmons, 2013; Wuetherick and Yu, 2016). Our initial presentations and writings of the SLaM model, however, predate the 4M model and the two models are distinct in various ways.

 


Leave a comment

Sources, Types, and Analysis of Data in SoTL

Written and compiled by Jennifer Friberg, Cross Endowed Chair in SoTL, and Professor of Communication Sciences & Disorders at Illinois State University

A few weeks ago, I published a blog titled “Study Design and Data Analysis in SoTL,” which provided a resource for viewing different types of research designs and their application for the scholarship of teaching and learning (SoTL). This resource was developed as a part of preparations for a two-day “intro to SoTL” workshop at the University of South Alabama. Today’s blog shares a related resource created for the same workshop series.

dataWhile research design is a really important consideration in the planning of SoTL projects, I would argue that an equally important consideration is the determination of the type(s) of data that could be collected to address an identified research purpose/question, as different types of data can provide different types of narratives to describe teaching and learning. The tables below explore eight different data types commonly utilized by individuals completing SoTL studies. Each describes a data type, talks about the data yielded from each type, indicates whether qualitative and/or quantitative analysis is possible for the data source in question, and provides extra information to consider possible to using any of these data sources. Please note that this is not an exhaustive list of possible data sources for SoTL research!

Should you wish to obtain a copy of this information in PDF form, please feel free to email me at jfribe@ilstu.edu. I’m happy to share!

Survey Data

Description Surveys collect data to reflect participant perceptions or knowledge about a particular phenomenon at one point in time.
Data Potentially Yielded Data generated via a survey are answers to specific questions drafted and administered. Survey questions can be closed ended (e.g., multiple choice answers or Likert-type scale data) or can be open ended in nature. Different question types yield different data.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Look for previously validated surveys to use as part of your study to increase validity/reliability of data collected.
  • For non-validated surveys, consider soliciting 2-3 expert reviewers to provide feedback re: survey content and format.
  • If survey is collecting indirect data (e.g., student perceptions), consider a plan to triangulate these data with a different, more directly objective source of information.

Interviews/Focus Groups

Description Considered a subset of survey research, these methods gather information about participant knowledge and feelings individually or with a group of people in a manner that allows (in some designs) for follow-up questions and non-standard data collection.  Interviews are generally conducted with a single person, while focus groups are group interviews.
Data Potentially Yielded Interactions occurring within interviews and focus groups are typically audio or video recorded. Orthographic transcriptions of these interactions can be created and analyzed to identify relevant trends across participants. Observations of specific behaviors might be quantified, as well, depending on the intent of the study’s design.
Quantitative Analysis Possible? Yes, but less frequent than qualitative analysis
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type Collection of interview and focus group data often causes extra human subjects review board scrutiny due to threats to confidentiality and/or anonymity. Consider how you will protect and explain protections for your study participants as part of your IRB development process.

Think Alouds

Description Think alouds are specific types of interviews where participants are asked to verbalize thoughts for internal cognitive processes in a sequential manner (e.g., how to complete a professionally-oriented task)
Data Potentially Yielded Like with interviews/focus groups, think alouds are typically audio or video recorded so that orthographic transcriptions of these interactions can be created and analyzed to identify relevant trends across participants.
Quantitative Analysis Possible? Yes, but less frequent than qualitative analysis
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type Similar to interviews and focus groups, collection of think aloud data often causes extra human subjects review board scrutiny due to threats to confidentiality and/or anonymity. Consider how you will protect and explain protections for your study participants.

Pre-/Post-Tests

Description Pre/post tests allow for collection of data to reflect changes resulting from some sort of intervention or experience over a pre-determined span of time.
Data Potentially Yielded Data collected is intended to reflect any changes (either positive or negative) resulting from an intervention or experience. Pre-/post-test data could be collected via a survey, reflection, or interview/focus group. The key here is that there are two (or more) sets of data to reflect differences across time.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Look for previously validated instruments to use for pre/post test designs.
  • For non-validated surveys, consider soliciting 2-3 expert reviewers to provide feedback re: survey content and format.
  • If pre/post test collects indirect data (e.g., student perceptions), consider a plan to triangulate these data with a different source.

Onlooker/Participant Observations

Description Specific, systematic observations conducted to collect behavioral data about participants within a teaching or learning context. In onlooker observations, the observer is not a part of the intervention/ experience. In participant observation, observers are active participants in the intervention/ experience.
Data Potentially Yielded Data collected from trained observers will quantify or describe the behaviors of participants at one or at multiple time frames. These data might be tallies of observed behaviors, descriptive notes describing behaviors, or time-managed tracking of behaviors in an environment.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • All observers should be carefully trained to collect data that reflects the intent of the study.
  • Observations can be made in real time or via videotaped sample.
  • Consider gathering inter-rater reliability data if more than one reviewer is operating within the context or project.

Course Assignments/Projects/Assessments

Description Course assignments, projects, or assessments are any tasks students complete as a part of your class which can be used to understand participant mastery of content or performance at a given point in an academic term or program. This might include: writings, journals, projects, online assignments, quizzes, tests, etc.
Data Potentially Yielded Data reflects a wide array of possibilities, but commonly would reflect participant knowledge and/or understanding of course content at a specific point in time during the course’s duration.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Any artifact that is part of a course you regularly teach or a course you have taught in the past can be used in a SoTL study. Consider the use of archival data to compare groups with and without a particular intervention or experience.
  • If you can no longer obtain consent from past students as they are gone from campus or you lack contact information for them, you can ask your IRB for a waiver of informed consent, so long as you have a plan to protect participant identity.

Written Student Reflections

Description Written student reflections are comprised of student thoughts and ideas presented that are expressed to demonstrate deep thinking and consideration (e.g., reflective journals).
Data Potentially Yielded Generally, data are journal entries or responses to specific reflection questions. Written data is analyzed to identify changes or trends across study participants.
Quantitative Analysis Possible? Yes, but less frequent than qualitative analysis
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Reflection data is almost always derived from some sort of prompt (e.g., journal prompt, reflective question). Craft these prompts carefully to ensure that you’re collecting the data most valuable for your study.
  • Analysis of written reflection data is almost always a qualitative endeavor. There are a variety of valid approaches to this sort of work, so consulting with a qualitative researcher if this is a new form of method for you is a good idea.

Visual Student Reflections

Description Visual student reflections are comprised of student thoughts and ideas presented that are expressed to demonstrate deep thinking and consideration (e.g., concept maps, drawings, figures, photos).
Data Potentially Yielded Visual reflection data provide representations of knowledge, skills, or learning at a given point in time to identify changes or trends across study participants.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Visual reflection data is almost always derived from some sort of classroom project or activity. Craft these experiences carefully to ensure that you’re collecting the data most valuable for your study.
  • Consider various visual data analysis methods as a lens for understanding your data.

 


1 Comment

Neil deGrasse Tyson and Community-Based SoTL Advocacy — Recommendations Inspired by a Popular Science Icon

Written by Jennifer Friberg, Cross Endowed Chair in SoTL, and Professor of Communication Sciences & Disorders at Illinois State University

Last week, the online version of The Chronicle of Higher Education published a piece by Vimal Patel titled “What Neil deGrasse Tyson Thinks Higher Ed Gets Wrong.” The article featured an interview with Tyson, a scientist and frequent media contributor/commentator, and discussed his perception that higher education is lacking a reward system (intrinsic or extrinsic) for communicating the work of researchers to the public. In his remarks, Tyson argued that teaching and public service are undervalued in most colleges and universities, relative to research. He posits that this fact contributes to public misunderstandings about science and research, as few researchers are actively and regularly engaged in sharing the findings of their scholarship outside their disciplines or institutions.

While the entirety of this interview focused on Tyson’s feelings toward science-based research, there were evident ties to the scholarship of teaching and learning (SoTL), as well, particularly in terms of Tyson’s thoughts on advocacy for research. As Tyson spoke of the need to regularly communicate research findings to the public, I was reminded of the idea that taking our SoTL work to the communities around our colleges and universities has been discussed by many as a vital, but often missing, component of SoTL advocacy. Tyson’s ideas tied directly to this notion. In sum, I noted three big takeaways from this article that inform opportunities for community-based SoTL advocacy:

Bring unexpected partners into discussions of SoTL. In his comments, Tyson shares that he interviewed singer Katy Perry on his radio show, much to the dismay of many who viewed Perry as a bad match for the typical science focus of Tyson’s shows. His response?

Why would [I] waste my time? She has more than 100 million Twitter followers. And if I can have a conversation with her about how science has touched her craft, then that brings science to her following. As far as I’m concerned, that adds value.

Why not look at SoTL in a similar fashion? Who can we bring to our craft to expand the reach and value of what we do? What groups of stakeholders can help spread the purpose and benefit of SoTL to others? Digging down, how do each of us identify targets for such advocacy in our own contexts and how might we connect with others for support and help?

Get better at communicating our SoTL research to the public.

Tyson argues that communicating research to the public is something that isn’t valued in higher education, particularly in the United States. He states:

Oxford has a tenured-professor line for the public understanding of science. I know of no such counterpart in the United States. Cambridge has a tenured-professor line for the public understanding of risk. Where is that here? These are [positions] where your ability to communicate is added to your academic chops.

I would argue that by virtue of our interest in SoTL, we are natural communicators. We are fluent in our disciplinary research but we are fluent in SoTL, as well. We translate to advocate, though this mostly occurs in our own institutional or disciplinary contexts. But, how many of us leave those contexts to enter the public sector? If we agree that groups outside our institutions might benefit from expanded SoTL advocacy, how do we get that message out? Might advancement centers, alumni networks, or research offices help? Should we do this work together in a cross-institutional manner? How might we engage established groups (i.e., ISSoTL) in a supportive or leading role for this work?

Keep it simple.

Tyson shared that part of his own development as a science commentator was understanding how his messages about science were most effectively shared. He reported that he believes his popularity in the media and with his followers lies in his ability to distill complex topics into digestible tidbits:

…the press can ask an academic question, and you can give an answer that you might give in a lecture hall. That’s not really the answer they want…I said, why don’t I just give them sound bites? So I went home and practiced in front of [my family]. They’d just randomly bark out questions about the universe, and I would deliver a two- or three-sentence reply. The anatomy of a soundbite has to be tasty, and you have to say, Wow, I’m glad I heard that. It has to…be so interesting that you want to tell someone else.

It’s likely true that once we endeavor to engage with stakeholders outside academia, we need to adapt how we communicate. As a speech-language pathology professor, I have often taught my students about a concept called “code switching,” wherein a speaker adapts how they deliver a message based on their audience. An example I frequently use to show how differently messages can be crafted via code switching is, “how would you ask the following people to open a window;”

When with a friend, you might ask “Dude, would you open the window?”

When with your younger brother, you might more directly say “open that window now!”

In the same room with your boss, you’d likely ask, “do you mind if I open that window?”

If we are seeking to share our SoTL with folks who aren’t academics, we need to learn to code switch a bit and use those communication skills I mentioned above in a slightly different way. We need to craft brief, summative, and engaging messages to appeal to folks unfamiliar with our work in an effort tot draw them in and hold their interest. We don’t need to give a forty minute paper. Rather, we need to make the case for how our SoTL work is important to them and how it might be in the future.

Blog Reference:

Patel, V. (2018). What Neil deGrasse Tyson thinks higher ed gets wrong. The Chronicle of Higher Education, 65(3). Retrieved from https://www.chronicle.com/article/What-Neil-deGrasse-Tyson/244522.