The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

SoTL, ER, and DBER: Thoughts Inspired by a Twitter Conversation

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences and Disorders at Illinois State University

A few weeks ago, I published a blog post titled “Finding the Goldilocks fit for your SoTL manuscript.” As always, I publicized the new blog post on my Office of the Cross Chair Twitter account (@ISU_SoTL). Who knew that a really great question from Erin Whitteck (@EWhitteck) would engender such a great conversation over the following days?

tweetstorm

Folks contributing to the subsequent tweet stream offered the suggestion that there is overlap between disciplinary-based educational research (DBER), SoTL, and educational research (ER), but that the lines between these types of inquiry could be a bit blurry. Questions were raised about rigor, methodological differences, and resources for better understanding. Since then, I’ve been pondering. To get us into the same semantic sandbox, consider the following definitions:

SoTL “involves the systematic study of teaching and/or learning and the public sharing and review of such work. ‘Study’ is broadly defined given disciplinary differences in epistemology and the need for interdisciplinary SoTL…SoTL focuses on teaching and learning at the college level, and is primarily classroom based. Ideally, SoTL also involves application and use” (McKinney, 2007, p. 10).

“ER is the scientific field of study that examines education and learning processes and the human attributes, interactions, organizations, and institutions that shape educational outcomes. Scholarship in the field seeks to describe, understand, and explain how learning takes place and how formal and informal contexts of education affect all forms of learning. Educational research embraces the full spectrum of rigorous methods appropriate to the questions being asked and also drives the development of new tools and methods” (AERA, 2018).

“DBER is grounded in the science and engineering disciplines and addresses questions of teaching and learning within those disciplines…DBER investigates teaching and learning in a discipline using a range of methods with deep grounding in the discipline’s priorities, worldview, knowledge, and practices…DBER is informed by and complementary to general [educational] research on human learning and cognition” (Singer, Neilsen, & Schweingruber, 2012, p. 9).

In response to the suggestion that there is overlap between SoTL, ER, and DBER, I believe that to be an undeniable truth. Each focuses on research on teaching and learning, serves to add knowledge to better understand educational processes, demands rigor, and has the potential for impact across contexts (e.g., micro, meso, macro, mega). SoTL, DBER, and ER also each purport to embrace a wide array of research approaches, including qualitative, quantitative, and mixed methods designs.

So, then, what about the differences? Here are a few that are important to consider:

  1. Both DBER and ER apply to K-12 research in addition to research in higher education. SoTL is focused on higher education.
  2. DBER is a form of ER, with a focus on science and engineering disciplines.
  3. The purpose of SoTL is to advance the practices of teaching and learning through systematic study and reflection (Larsson, Martensson, Price, & Roxa, 2017). The purpose of ER and DBER is to create generalizable knowledge about teaching and learning (Felten, 2015), though it should be noted that DBER scholars acknowledge a need to translate DBER findings to classroom practice, in line with SoTL (Singer, Neilsen, & Schweingruber, 2012).
  4. A common tenant of SoTL is that scholars study their unique learning contexts to better understand their teaching and/or their students’ learning. Most ER removes the investigator from the context being studied.
  5. While ER and DBER seek to create generalizable findings, most SoTL is not inherently generalizable as it often studies a single learning context and might study a small(ish) number of individuals. Rather, SoTL should be inherently replicable through the explanation of a systematic approach to investigation that is reported when results are disseminated. SoTL seeks to build generalizability over time as different constructs are studied in different places by different people at different times.
  6. SoTL embraces a “big tent” philosophy with a wide array of disciplines and diverse approaches to inquiry recognized as making important contributions to research on teaching and learning. As ER and DBER typically focus on education or STEM fields, theories, methods, and practices for these disciplines are typically utilized in those types of inquiry.

ER, DBER, and SoTL are all valuable forms of teaching and learning research. While there is overlap between and across these categories of research, they are not competitors. They exist on a continuum that encourages scholarly approaches teaching and further research on teaching and learning. I would argue that it is the interpretation of the similarities and differences of SoTL, ER, and DBER that friction might emerge, as we typically consider research through our own disciplinary lenses. That might be topic for a future blog all on its own…

So, Erin, I’ll try to answer Twitter question from earlier in November that launched this discussion: “what is the difference between a disciplinary SoTL journal and a DBER journal?” Honestly, there may not be a difference. In some fields, SoTL and DBER might both be published in the same journal. In others, it might be one or the other. I’d suggest that you look at the aims and scope statements for your discipline’s SoTL and DBER journals. Identify which aligns with the work you’ve done in terms of purpose (e.g., add or apply knowledge). If you’re not sure, editors LOVE getting emails from prospective contributors. I really mean this! Send an abstract of your work and ask if it’s suitable for their journal or ask a question or two to guide your efforts. Good luck!

Blog References:

American Educational Research Association. (2018). What is educational research? Downloaded from http://www.aera.net/About-AERA/What-is-Education-Research.

Felten, P. (2015). Principles of good practice in SoTL. Teaching & Learning Inquiry, 1(1), pp. 121-125.

Larsson, M., Martensson, K., Priace, L. & Roxa, T. (2017). Constructive friction? Exploring patters between educational research and the scholarship of teaching and learning. Paper presented at the 2nd EuroSoTL Conference, Lund, Sweden.

McKinney, K. (2007). Enhancing learning through the scholarship of teaching and learning: The challenges and joys of juggling. Anker Publishing: Boston, MA.

Singer, S. R., Nielsen, N. R., & Schweingruber, H. A. (Eds.). (2012). Discipline-based education research: understanding and improving learning in undergraduate science and engineering. National Academies Press: Washington, D.C.

Advertisements


Leave a comment

Finding the Goldilocks fit for your SoTL manuscript: It’s a question of content, voice, and application!

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences and Disorders at Illinois State University (contact email: jfribe@ilstu.edu)

As is the case with disciplinary research, SoTL research is carried out carefully and systematically. Data is analyzed, results are presented, and a compelling case is made for the implications of the outcomes of SoTL research process. For those of us for whom a peer-reviewed journal article is the “currency” of academic productivity, we think about where we might eventually send our work for review and (hopefully!) publication throughout our project’s life. We search lists of SoTL publication outlets seeking the Goldilocks “fit” for our research, carefully reviewing the aims, scopes, and missions of SoTL journals as part of this process. As these efforts unfold, there is a foundational question that must be asked as part of the search for a journal “home” for your SoTL work: Does my SoTL best fit in a disciplinary journal or a cross-disciplinary journal?

To make sure we are all on the same page semantically, I’d define a disciplinary SoTL journal as one that focuses primarily on one discipline. Teaching and Learning in Communication Sciences and Disorders is one that is a great example of this, with its focus on SoTL for the connected disciplines of speech-language pathology and audiology. Teaching and Learning Inquiry would be an ideal example of a cross-disciplinary SoTL journal, as manuscripts selected for publication potentially apply to a variety of disciplines across the academic spectrum.

The question of disciplinary versus cross-disciplinary fit has to do (mainly) with the potential reach for your work. For instance, if you conduct a rigorous SoTL project to understand how art history students’ learning is impacted through study abroad experiences in Italian museums, it’s possible that your findings might have primary interest and impact within the discipline of art history. As such, a journal like Art History Pedagogy & Practice would be a wonderful outlet for your work. A study on intrapersonal learning as a result of students’ involvement with an array of campus student organizations might have a broader disciplinary appeal, with publication in the cross-disciplinary Journal of the Scholarship of Teaching and Learning one potential outcome.

That said, it’s often how a manuscript is written that truly influences its fit for publication. With this in mind, three primary considerations become evident:

  • The content of your manuscript is extremely important. Is the topic being explored centered on questions from a single discipline? Or, might the content of your paper be of interest to people representing a variety of disciplines and contexts?
  • Your writing voice is also critical. When you constructed your manuscript, did you use accessible terminology or did you employ disciplinary jargon to best make your points?
  • How have you described the potential applications of your work? Did you tie your findings to uses and impacts in one discipline or did you make an effort to extend your research outcomes to a variety of fields and contexts?

The decision tree below operationalizes the notions of content, voice, and application through the lens that the more linguistically accessible and contextually inclusive your manuscript seeks to be, the more likely it is to find a fit in a cross-disciplinary SoTL journal.

SoTL decision tree

I have one last thought for your consideration. Some SoTL is simply so focused on one discipline that its contributions to the pedagogical content knowledge of that discipline must be honored with publication in a disciplinary journal. Similarly, some SoTL cannot be tied to only one discipline, or perhaps it’s so applicable to other disciplines that publishing in a cross-disciplinary outlet is its best fit. Thus, SoTL is not “better” or “worse” if its published in a disciplinary rather than a cross-disciplinary journal — or vice versa. Rather, it’s knowing where your SoTL belongs that helps it to have value to your audience. 

 


Leave a comment

The SLaM Model of Applying SoTL In and Beyond One Classroom

Written by Kathleen McKinney, Illinois State University, Emeritus and Jennifer Friberg, Illinois State University 

slamIn this blog post we share a model for the application of scholarship of teaching and learning findings in and beyond the individual classroom level. The model, named SLaM, is detailed in the Introduction chapter of our edited book, Applying the Scholarship of Teaching and Learning Beyond the Individual Classroom (Indiana University Press, 2019, in press). The focus of that volume is on SoTL and its application beyond one classroom but the SLaM model is about application at any level. We define SoTL using both our institutional definition, ‘the systematic reflection/study of teaching and learning made public,’ as well as key characteristics as practitioner, action reflection/research that is usually about the instructor/researchers’ own students and/or students in their discipline and is most often at the local level. We understand application as the use of SoTL research findings and implications to design, change, intervene, make decisions, etc., primarily in institutions and disciplines, to enhance teaching and student learning.

The SLaM model is an outgrowth of our early discussions of application at various levels (e.g., Friberg & McKinney, 2015, 2016; McKinney 2003, 2007, 2012).[1] We then organized and built on those ideas, as we wrote for and edited our latest book, to create the SLaM model. The model uses three questions to conceptualize, categorize, and understand the use of SoTL results/knowledge in applications to teaching and learning. We briefly note these here but a more detailed discussion, diagram, and examples of the model can be found in our Introduction to our edited book (see endnote 1 below for the citation for the model).

  1. What is the source of the SoTL that is applied? The “S” in our SLaM framework is connected to identifying the source(s) of SoTL findings being applied. SoTL research results that are applied at various levels may be from the teacher’s original scholarship of teaching and learning studies, SoTL work by colleagues, the synthesis of presented or published SoTL research in the discipline/institution/larger SoTL field, or some combination of these sources of SoTL results and implications.
  2. At what level(s) are the data/results/implications applied? There are numerous levels (the “L” in our framework) at which SoTL findings and implications could be applied to positively impact teaching and learning. These levels include the individual classroom, course/module, program, department, college, co-curricular, institutional, disciplinary, multi-institutional, and multi-disciplinary levels.
  3. What mechanisms or processes are used (or could be used) to apply the SoTL data/results/implications to new areas or contexts at various levels? The “M” in our SLaM framework represents the many mechanisms that exist or could be created that can be used as processes for novel applications of SoTL findings. A few examples include assessment, quality assurance, course/program design or redesign, accreditation, budget development, strategic planning, faculty/staff development, interdisciplinary initiatives, and graduate student training.

In our forthcoming edited book, eleven examples of the application of SoTL are described; two in our Introduction and nine in the contributed chapters. We briefly summarize three of these examples of applications and their fit with our model here. First, Brent Oliver, Darlene Chalmers, and Mary Goitom of Mount Royal University in Canada in their chapter, “Reflexivity in the Field: Applying Lessons Learned from a Collaborative Scholarship of Teaching and Learning Study Exploring the Use of Reflexive Photography in Field Education” use findings and implications from face-to face interviews with students from multiple institutions (source). They apply what they learned at the course, program and department levels using curricular reform, program review and accreditation (mechanisms). They are planning additional applications in a new interdisciplinary fellowship program and via faculty development programs.

Another example comes from Belgium. In the chapter, “Feedback First Year”- A Critical Review of the Strengths and Shortcomings of a Collective Pedagogical Project,” Dominique Verpoorten, Laurent Leduc, Audrey Mohr, Eléonore Marichal, Dominique Duchâteau, and Pascal Detroz describe their sources of SoTL findings: SoTL literature on feedback practices as well as original data from interviews with members of the faculty participating in SoTL staff development programs, observations and diaries of advisers, minutes of meetings, and descriptive templates of project outcomes. Levels of application included individual courses, faculties/departments (group of courses; program), and institution. The mechanisms they used for application were specific course re-design tasks (designing feedback activities by faculty participants), a variety of course interventions, and sharing results in departments via meetings and plenaries.

Finally, contributors Claire Vallotton, Gina A. Cook, Rachel Chazan-Cohen, Kalli B. Decker, Nicole Gardner-Neblett, Christine Lippard, and Tamesha Harewood share their SoTL applications in “The Collaborative for Understanding the Pedagogy of Infant/toddler Development: A Cross-University, Interdisciplinary Effort to Transform a Field through SoTL.” Their project used implications from past SoTL literature, reflection, and original SoTL studies on multiple campuses (sources) at the course, program, department and disciplinary levels. The application mechanism was a cross-institutional, collaborative group of scholars (CUPID) where participants shared resources, conducted research, and disseminated work via conferences, workshops, publications, meetings.

We hope readers of this blog post will take a look at the details of the SLaM model and the interesting projects and applications from around the globe presented in the edited volume. We welcome feedback on the model and hope others will find it useful in their SoTL research and applications.

Blog References

Friberg, Jennifer C., and Kathleen McKinney. 2016. “Creating Opportunities for Institutional and Disciplinary SoTL Advocacy and Growth.” Presentation. SoTL Commons Conference, Savannah, GA, USA.

Friberg, Jennifer C., and Kathleen McKinney. 2015. “Strengthening SoTL at the Institutional and Disciplinary Levels.” Poster presentation. EuroSoTL, Cork, Ireland.

McKinney, Kathleen. 2012. “Making a Difference: Applying SoTL to Enhance Learning.” The Journal of the Scholarship of Teaching and Learning 12(1): 1-7.

McKinney, Kathleen. 2007. Enhancing Learning through the Scholarship of Teaching and Learning: The Challenges and Joys of Juggling. San Francisco: Jossey-Bass.

McKinney, Kathleen. 2003. “Applying the Scholarship of Teaching and Learning: How Can We Do Better?” The Teaching Professor August-September:1,5,8.

 

[1] As discussed in the Introduction to our edited book, the SlaM model overlaps slightly with the 4M model (Poole and Simmons, 2013; Wuetherick and Yu, 2016). Our initial presentations and writings of the SLaM model, however, predate the 4M model and the two models are distinct in various ways.

 


Leave a comment

Sources, Types, and Analysis of Data in SoTL

Written and compiled by Jennifer Friberg, Cross Endowed Chair in SoTL, and Professor of Communication Sciences & Disorders at Illinois State University

A few weeks ago, I published a blog titled “Study Design and Data Analysis in SoTL,” which provided a resource for viewing different types of research designs and their application for the scholarship of teaching and learning (SoTL). This resource was developed as a part of preparations for a two-day “intro to SoTL” workshop at the University of South Alabama. Today’s blog shares a related resource created for the same workshop series.

dataWhile research design is a really important consideration in the planning of SoTL projects, I would argue that an equally important consideration is the determination of the type(s) of data that could be collected to address an identified research purpose/question, as different types of data can provide different types of narratives to describe teaching and learning. The tables below explore eight different data types commonly utilized by individuals completing SoTL studies. Each describes a data type, talks about the data yielded from each type, indicates whether qualitative and/or quantitative analysis is possible for the data source in question, and provides extra information to consider possible to using any of these data sources. Please note that this is not an exhaustive list of possible data sources for SoTL research!

Should you wish to obtain a copy of this information in PDF form, please feel free to email me at jfribe@ilstu.edu. I’m happy to share!

Survey Data

Description Surveys collect data to reflect participant perceptions or knowledge about a particular phenomenon at one point in time.
Data Potentially Yielded Data generated via a survey are answers to specific questions drafted and administered. Survey questions can be closed ended (e.g., multiple choice answers or Likert-type scale data) or can be open ended in nature. Different question types yield different data.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Look for previously validated surveys to use as part of your study to increase validity/reliability of data collected.
  • For non-validated surveys, consider soliciting 2-3 expert reviewers to provide feedback re: survey content and format.
  • If survey is collecting indirect data (e.g., student perceptions), consider a plan to triangulate these data with a different, more directly objective source of information.

Interviews/Focus Groups

Description Considered a subset of survey research, these methods gather information about participant knowledge and feelings individually or with a group of people in a manner that allows (in some designs) for follow-up questions and non-standard data collection.  Interviews are generally conducted with a single person, while focus groups are group interviews.
Data Potentially Yielded Interactions occurring within interviews and focus groups are typically audio or video recorded. Orthographic transcriptions of these interactions can be created and analyzed to identify relevant trends across participants. Observations of specific behaviors might be quantified, as well, depending on the intent of the study’s design.
Quantitative Analysis Possible? Yes, but less frequent than qualitative analysis
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type Collection of interview and focus group data often causes extra human subjects review board scrutiny due to threats to confidentiality and/or anonymity. Consider how you will protect and explain protections for your study participants as part of your IRB development process.

Think Alouds

Description Think alouds are specific types of interviews where participants are asked to verbalize thoughts for internal cognitive processes in a sequential manner (e.g., how to complete a professionally-oriented task)
Data Potentially Yielded Like with interviews/focus groups, think alouds are typically audio or video recorded so that orthographic transcriptions of these interactions can be created and analyzed to identify relevant trends across participants.
Quantitative Analysis Possible? Yes, but less frequent than qualitative analysis
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type Similar to interviews and focus groups, collection of think aloud data often causes extra human subjects review board scrutiny due to threats to confidentiality and/or anonymity. Consider how you will protect and explain protections for your study participants.

Pre-/Post-Tests

Description Pre/post tests allow for collection of data to reflect changes resulting from some sort of intervention or experience over a pre-determined span of time.
Data Potentially Yielded Data collected is intended to reflect any changes (either positive or negative) resulting from an intervention or experience. Pre-/post-test data could be collected via a survey, reflection, or interview/focus group. The key here is that there are two (or more) sets of data to reflect differences across time.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Look for previously validated instruments to use for pre/post test designs.
  • For non-validated surveys, consider soliciting 2-3 expert reviewers to provide feedback re: survey content and format.
  • If pre/post test collects indirect data (e.g., student perceptions), consider a plan to triangulate these data with a different source.

Onlooker/Participant Observations

Description Specific, systematic observations conducted to collect behavioral data about participants within a teaching or learning context. In onlooker observations, the observer is not a part of the intervention/ experience. In participant observation, observers are active participants in the intervention/ experience.
Data Potentially Yielded Data collected from trained observers will quantify or describe the behaviors of participants at one or at multiple time frames. These data might be tallies of observed behaviors, descriptive notes describing behaviors, or time-managed tracking of behaviors in an environment.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • All observers should be carefully trained to collect data that reflects the intent of the study.
  • Observations can be made in real time or via videotaped sample.
  • Consider gathering inter-rater reliability data if more than one reviewer is operating within the context or project.

Course Assignments/Projects/Assessments

Description Course assignments, projects, or assessments are any tasks students complete as a part of your class which can be used to understand participant mastery of content or performance at a given point in an academic term or program. This might include: writings, journals, projects, online assignments, quizzes, tests, etc.
Data Potentially Yielded Data reflects a wide array of possibilities, but commonly would reflect participant knowledge and/or understanding of course content at a specific point in time during the course’s duration.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Any artifact that is part of a course you regularly teach or a course you have taught in the past can be used in a SoTL study. Consider the use of archival data to compare groups with and without a particular intervention or experience.
  • If you can no longer obtain consent from past students as they are gone from campus or you lack contact information for them, you can ask your IRB for a waiver of informed consent, so long as you have a plan to protect participant identity.

Written Student Reflections

Description Written student reflections are comprised of student thoughts and ideas presented that are expressed to demonstrate deep thinking and consideration (e.g., reflective journals).
Data Potentially Yielded Generally, data are journal entries or responses to specific reflection questions. Written data is analyzed to identify changes or trends across study participants.
Quantitative Analysis Possible? Yes, but less frequent than qualitative analysis
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Reflection data is almost always derived from some sort of prompt (e.g., journal prompt, reflective question). Craft these prompts carefully to ensure that you’re collecting the data most valuable for your study.
  • Analysis of written reflection data is almost always a qualitative endeavor. There are a variety of valid approaches to this sort of work, so consulting with a qualitative researcher if this is a new form of method for you is a good idea.

Visual Student Reflections

Description Visual student reflections are comprised of student thoughts and ideas presented that are expressed to demonstrate deep thinking and consideration (e.g., concept maps, drawings, figures, photos).
Data Potentially Yielded Visual reflection data provide representations of knowledge, skills, or learning at a given point in time to identify changes or trends across study participants.
Quantitative Analysis Possible? Yes
Qualitative Analysis Possible? Yes
Miscellaneous Thoughts on Data Type
  • Visual reflection data is almost always derived from some sort of classroom project or activity. Craft these experiences carefully to ensure that you’re collecting the data most valuable for your study.
  • Consider various visual data analysis methods as a lens for understanding your data.

 


1 Comment

Neil deGrasse Tyson and Community-Based SoTL Advocacy — Recommendations Inspired by a Popular Science Icon

Written by Jennifer Friberg, Cross Endowed Chair in SoTL, and Professor of Communication Sciences & Disorders at Illinois State University

Last week, the online version of The Chronicle of Higher Education published a piece by Vimal Patel titled “What Neil deGrasse Tyson Thinks Higher Ed Gets Wrong.” The article featured an interview with Tyson, a scientist and frequent media contributor/commentator, and discussed his perception that higher education is lacking a reward system (intrinsic or extrinsic) for communicating the work of researchers to the public. In his remarks, Tyson argued that teaching and public service are undervalued in most colleges and universities, relative to research. He posits that this fact contributes to public misunderstandings about science and research, as few researchers are actively and regularly engaged in sharing the findings of their scholarship outside their disciplines or institutions.

While the entirety of this interview focused on Tyson’s feelings toward science-based research, there were evident ties to the scholarship of teaching and learning (SoTL), as well, particularly in terms of Tyson’s thoughts on advocacy for research. As Tyson spoke of the need to regularly communicate research findings to the public, I was reminded of the idea that taking our SoTL work to the communities around our colleges and universities has been discussed by many as a vital, but often missing, component of SoTL advocacy. Tyson’s ideas tied directly to this notion. In sum, I noted three big takeaways from this article that inform opportunities for community-based SoTL advocacy:

Bring unexpected partners into discussions of SoTL. In his comments, Tyson shares that he interviewed singer Katy Perry on his radio show, much to the dismay of many who viewed Perry as a bad match for the typical science focus of Tyson’s shows. His response?

Why would [I] waste my time? She has more than 100 million Twitter followers. And if I can have a conversation with her about how science has touched her craft, then that brings science to her following. As far as I’m concerned, that adds value.

Why not look at SoTL in a similar fashion? Who can we bring to our craft to expand the reach and value of what we do? What groups of stakeholders can help spread the purpose and benefit of SoTL to others? Digging down, how do each of us identify targets for such advocacy in our own contexts and how might we connect with others for support and help?

Get better at communicating our SoTL research to the public.

Tyson argues that communicating research to the public is something that isn’t valued in higher education, particularly in the United States. He states:

Oxford has a tenured-professor line for the public understanding of science. I know of no such counterpart in the United States. Cambridge has a tenured-professor line for the public understanding of risk. Where is that here? These are [positions] where your ability to communicate is added to your academic chops.

I would argue that by virtue of our interest in SoTL, we are natural communicators. We are fluent in our disciplinary research but we are fluent in SoTL, as well. We translate to advocate, though this mostly occurs in our own institutional or disciplinary contexts. But, how many of us leave those contexts to enter the public sector? If we agree that groups outside our institutions might benefit from expanded SoTL advocacy, how do we get that message out? Might advancement centers, alumni networks, or research offices help? Should we do this work together in a cross-institutional manner? How might we engage established groups (i.e., ISSoTL) in a supportive or leading role for this work?

Keep it simple.

Tyson shared that part of his own development as a science commentator was understanding how his messages about science were most effectively shared. He reported that he believes his popularity in the media and with his followers lies in his ability to distill complex topics into digestible tidbits:

…the press can ask an academic question, and you can give an answer that you might give in a lecture hall. That’s not really the answer they want…I said, why don’t I just give them sound bites? So I went home and practiced in front of [my family]. They’d just randomly bark out questions about the universe, and I would deliver a two- or three-sentence reply. The anatomy of a soundbite has to be tasty, and you have to say, Wow, I’m glad I heard that. It has to…be so interesting that you want to tell someone else.

It’s likely true that once we endeavor to engage with stakeholders outside academia, we need to adapt how we communicate. As a speech-language pathology professor, I have often taught my students about a concept called “code switching,” wherein a speaker adapts how they deliver a message based on their audience. An example I frequently use to show how differently messages can be crafted via code switching is, “how would you ask the following people to open a window;”

When with a friend, you might ask “Dude, would you open the window?”

When with your younger brother, you might more directly say “open that window now!”

In the same room with your boss, you’d likely ask, “do you mind if I open that window?”

If we are seeking to share our SoTL with folks who aren’t academics, we need to learn to code switch a bit and use those communication skills I mentioned above in a slightly different way. We need to craft brief, summative, and engaging messages to appeal to folks unfamiliar with our work in an effort tot draw them in and hold their interest. We don’t need to give a forty minute paper. Rather, we need to make the case for how our SoTL work is important to them and how it might be in the future.

Blog Reference:

Patel, V. (2018). What Neil deGrasse Tyson thinks higher ed gets wrong. The Chronicle of Higher Education, 65(3). Retrieved from https://www.chronicle.com/article/What-Neil-deGrasse-Tyson/244522.

 


1 Comment

Study Design and Data Analysis in SoTL

Written and compiled by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences & Disorders at Illinois State University

In June, I visited the University of South Alabama (USA) and worked with Raj Chaudhury and Sue Mattson to get a group of faculty started with their year-long SoTL Academy efforts. Approximately 30 faculty from across USA’s campus came together to learn about SoTL and plan a SoTL project. We spent two days together in workshops and consultations and all participants left with a draft plan for SoTL work they hoped to conduct this current academic year.

This was the second year I was able to join the USA crew for this two-day educational and research development event. Sue and I agreed that a resource that would be valuable for the USA faculty for the second iteration of the SoTL Academy would be something that helped social science-oriented researchers see how SoTL might dovetail with concepts and ideas they already understood well. Thus, the following grids focused on descriptive, correlational, and experimental/quasi-experimental design were drafted and used in discussions about how SoTL might look like participants’ own disciplinary research — and how it might not. This resource is being shared here now, in the hopes that others might find this information valuable, as well.

Should you wish to obtain a copy of this information in PDF form, please feel free to email me at jfribe@ilstu.edu. I’m happy to share!

Descriptive Research
Description of Study Design Descriptive research characterizes a group of people, a context, or a phenomenon. These studies do not seek to establish a causal relationship; rather, they provide information about “what is” occurring or being observed regarding the focus of study.

Descriptive studies include observational, case study, and survey methods.

Exemplar SoTL Projects
  • Survey students’ re: practices in using print vs. online textbooks to support learning.
  • Observe how students’ use of technology in the classroom impacts attention span.
  • Study high achieving students in a course to predict practices/variables of success to share with future students.
Qualitative Analysis Options Qualitative data in a descriptive study is reported as narrative, reflection, open-ended response, field note, etc. Such data will need to be further analyzed for themes, categories, or patterns.

Common qualitative approaches in descriptive SoTL research include: case studies, action research processes, analytic induction, ethnography, comparative analysis, frame analysis, grounded theory, and interpretive phenomenology, among others.

Quantitative Analysis Options Quantitative data in a descriptive study is often reported in the form of descriptive statistics (e.g., mean, median, mode) along with standard deviations. Statistics might be used here, depending on the data collected and the topic being studied.

These data might emerge from test scores, grades on a course assignment or project, survey data, or frequency data.

 

Correlational Research
Description of Study Design Correlational research seeks to determine whether a relationship exists between two or more variables, but cannot determine if one variable causes another. Variables aren’t manipulated; rather, they are observed to determine any relationship that might exist between them.

Note that some sources identify correlational research as a quantitative-only subset of descriptive research, as some descriptive research might suggest a correlation found via grounded theory or other qualitative methods of research.

Exemplar SoTL Projects
  • Determine the relationship between number of hours studying and success on a quiz/test.
  • Identify whether there is a link between the use of peer editing and performance on a writing assignment.
  • Understand whether the use of social media helps students to summarize course content effectively.
Qualitative Analysis Options Qualitative data analysis is not undertaken for correlational research, as numerical data is needed to calculate a correlation coefficient.
Quantitative Analysis Options Correlational research is a quantitative method of inquiry. Correlation can only be determined for quantifiable data. These are data in which numbers are meaningful, usually quantities of some sort. It cannot be used for purely categorical data, such as gender, brands purchased, or favorite color.

Statistics are used to determine a correlation coefficient to identify positive, negative, or zero correlation. One thing to keep in mind is that any identified correlation does not mean that one variable caused the other to react. Instead, correlations simply define that a relationship exists.

 

Experimental/Quasi-Experimental Research
Description of Study Design Experimental and quasi-experimental research designs seek to manipulate one variable and control all others to investigate cause/effect relationships. All participants are assigned to either a control or experimental group. An intervention is applied to the experimental group. The control group has no intervention applied.

The key difference between experimental and quasi-experimental designs is the concept of randomization. If participants are assigned to control and experimental groups randomly, the research design is experimental. Non-random group assignment yields a quasi-experimental research design. True experimental research is considered the gold standard of research by many researchers, because random group assignment leads to optimal internal validity. In situations where random group assignment is not possible or ethical, quasi-experimental designs offer an alternative that allows the research to continue and still produce valid results.

Almost no SoTL qualifies as truly experimental in nature due to inherent ethical and logistical characteristics of SoTL that makes this type of research difficult to conduct (e.g., true randomization). One of the most common quasi-experimental designs for SoTL research is the pre-test/post-test with no control group design.

Exemplar SoTL Projects
  • Does the use of simulated patients help nursing students improve observational skills?
  • Do architecture students who initially design structures by hand understand the concept of space more deeply?
  • Do history students exposed to guided reading demonstrate a deeper understanding of historical imagination?
Qualitative Analysis Options Experimental and quasi-experimental designs may yield data that is descriptive (e.g., surveys, interviews, observations) that require qualitative analyses. Similar to information provided above for descriptive research, any qualitative data will need to be further analyzed for themes, categories, or patterns.

Common qualitative approaches to data analysis in SoTL include: case study, action research processes, analytic induction, ethnography, comparative analysis, frame analysis, grounded theory, and interpretive phenomenology, among others.

Quantitative Analysis Options Experimental design lends itself to more straightforward and simpler types of statistical analysis. Primarily due to the lack of randomization, quasi-experimental studies usually require more advanced statistical procedures. Quasi-experimental designs may also utilize surveys, interviews, and observations which may further complicate the data analysis.

Quantitative analysis requires several steps. First numeric data is assigned a level (nominal, ordinal, interval, or ratio). Next, descriptive statistics are calculated for data (e.g., means, standard deviations). For some studies, descriptive statistics may be adequate; however, if you want to make inferences or predictions about your population, inferential statistics (e.g., t-test, ANOVA, regression) may be indicated.

Blog References:

Bishop-Clark, C. & Dietz-Uhler, B. (2012). Engaging in the scholarship of teaching and learning: A guide to the process and how to develop a project from start to finish.        Stylus: Sterling, VA.

Campbell, D. T. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Cengage: Boston.

Cresswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Sage: Thousand Oaks, CA.

Gurung, R. A. R. & Wilson, J. H. (Eds.). (2014). Doing the scholarship of teaching and learning: Measuring systematic changes to teaching and improvements in learning. Jossey-Bass: San Francisco.

 

 


Leave a comment

An Idea for the First Days of the Fall Term – Share SoTL with Your Students!

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences & Disorders at Illinois State University

Screen Shot 2018-08-13 at 2.01.02 PMBack in April, I wrote a blog regarding the impact of SoTL that was inspired by my reading of this article by Nancy Chick. I’ve thought a lot about the notion of impact since that time, considering how we encourage changes in teaching and learning as a result of our SoTL efforts. I’ve engaged in conversations with numerous colleagues (on my campus and at others) about how they adapt their teaching praxis in the presence of good evidence to do so. As a result of these exchanges, I feel at least somewhat confident that our SoTL work IS making change; however, these conversations have left me wondering if we aren’t missing a huge opportunity to truly increase the impact of our SoTL efforts and outcomes. In no conversation about how SoTL has changed our teaching and learning did anyone I spoke with discuss sharing SoTL with their students. There was discussion about changing course content, assessment, or management, but each of these things was described as occurring in relative solitude as part of next generation course design.

I find it curious that we study our students to understand the components of meaningful learning and teaching experiences, but in doing so, (at least some of us) miss out on purposeful sharing of SoTL outcomes with our students so they can make changes to THEIR praxis as learners. We have generated so much evidence that shows us how students learn (and learn well!). They should have access to this information and it’s my strong opinion that we should help facilitate that access.

Here are a few thoughts as to how we might be more purposeful in bringing students into the SoTL loop — feel free to share other thoughts and ideas in the comments below:

  • Share information about relevant, evidence-based learning strategies as part of your class. Many course instructors have “syllabus review day” during the first course meeting of a new term. While there are great suggestions about alternative ideas for that first course meeting circulating social media this time of year, perhaps a focus on successful learning strategies might be a worthy way to spend that first class together. Share what you know about evidence-based learning strategies that might be useful for your students in your context. Let them know that you’re a resource and would be interested in answering questions about evidence-based strategies for learning. Provide resources for students to access this information themselves.
  • Mediate! Tell your students WHY you’ve designed your course or assignment or assessment in the manner that you have – share your evidence! I do this frequently with my students and have found that if I can provide the rationale for what they are doing, and that research has shown a pedagogical approach to be impactful, I have more buy-in and (anecdotally) more active engagement in the task(s) at hand.
  • Share what others in your discipline have identified as evidence-based learning strategies for emerging professionals. How do sociologists develop a sociological imagination? How do mathematicians generalize concepts to varied contexts? How do historians read a text and assess primary sources? How do speech-language pathologists, nurses, or dieticians transfer theory to clinical practice? SoTL has helped us understand these discipline-specific phenomena. Unlock these connections for students to visualize a path toward professional practice that is grounded in evidence.
  • Use your social media smartly. Does your university have a Twitter or Instagram account where you could populate content about evidence-based ways to learn or study? Can you feature links to and/or summaries of the work of SoTL scholars on your campus to highlight what you know about learning in your own institutional context? Can you manage (or co-manage) an account yourself that does this?
  • Offer to guest “lecture” about evidence-based learning at a meeting of a student organization tied to your discipline or some other movement. Talk to students about research on teaching and learning and how outcomes of such research can support their work as students. There is evidence that out-of-class learning through student organizations, service learning, and civic engagement have efficacy. Let students know the benefits of these efforts!
  • Take care in making assumptions about what students know. Based on the fact that our students are enrolled at our colleges/universities, it would be easy to think that they have unlocked the mysteries of learning deeply and well. They wouldn’t be college students if they hadn’t accomplished that, right? I’m not convinced this is actually the case. I have spoken to numerous students who engage in low utility learning strategies to master material who are frustrated with their lack of ability to make connections and applications across topics and classes. My bias? Assume that your students would be interested to know more about teaching and learning until you know differently.

Writing on a similar topic, McKinney (2012, p. 3) suggested the following strategies for bringing students to SoTL, specifically by discussing the “how” and the “why” of SoTL research and findings emerging from such inquiry:

  • Make SoTL public at conferences students attend and in publications students read.
    Create a local SoTL journal or newsletter aimed specifically at college students at
    your institution or a national/international one for students in a specific major or
    discipline.
  • Use SoTL publications as required readings in courses where they are appropriate
    such as a disciplinary/department new majors‟ orientation class, a research methods course, a capstone course, or a professional socialization course.
    Facilitate and invite students to sessions on learning on campus that share, and
    discuss implications of, local SoTL results.
  • Volunteer to create a session at your disciplinary meetings focusing on key SoTL
    results and explicitly involve and invite students.
  • Add a section of relevant SoTL study results and any implications for students to
    your department website within the web pages for students.
  • Help organize a panel where SoTL researchers present and lead a discussion with
    students at a meeting of your student disciplinary/department club.
  • Include in your courses, when appropriate, reflective and meta-cognitive
    assignments that help students relate SoTL literature and findings to their own
    learning opportunities and behaviors.

 

Blog References:

McKinney, K. (2012). Increasing the impact of SoTL: Two sometimes neglected opportunities. International Journal for the Scholarship of Teaching and Learning, 6(1).