The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Theoretical Pattern-Matching in SoTL: SoTL Methods Series #1

This blog serves as the beginning of a four-week focus on unique research methods for SoTL work. Enjoy, and please feel free to write to our guest bloggers with any feedback or questions! -Jen Friberg, blog editor

Written by: Bill Anderson, Associate Professor of Family and Consumer Sciences at Illinois State University (jander2@ilstu.edu)

I recently completed a SoTL project where I was hoping to create vicarious, but meaningful, applications of classroom learning, in this case, foundational theories of the human development field. In an attempt to accomplish this, I utilized interrupted case studies (ICS), a progressive disclosure of information viewed as problem-based learning over time. Over an eight week period following a pre-test application, students viewed a longitudinal series of interviews as an ICS. This series followed several participants from the time they were seven years old in 1964, revisiting them every seven years until age 56 in 2013. During the process, and using the assumptions, concepts, and language of assigned developmental theorists, students described and applied relevant theoretical positions to anticipate growth and change as this collection of real lives progressed. Their work was submitted in weekly reflective essays. At the end of the eight-week assignment, post-test results indicated that the method was quite successful but told me nothing further. The post-test increase could simply be the result of memorizing the material. Therefore, pattern-matching was applied to further examine those results.

patternPattern-matching is a less-known, but dependable, procedure for theory testing with case-studies and is regularly recommended for reconciling mixed methods and data sources in case study research, and to boost the rigor of the study. The overarching goal is to explain relationships between key points, in this case the pre/post results, by comparing an identified theoretical pattern with an observed pattern.

The previously mentioned weekly student essays were utilized as the observed pattern. These included descriptions of their assigned interview participants, appraisals of their most recent developmental predictions for this person, and their expectations for the next seven years. The essays were coded line-by-line to determine the degree of matching to a predetermined theoretical pattern. In this case, Bloom’s Revised Taxonomy (Anderson & Krathwohl, 2001) was utilized as coding categories as follows: 1 – Remembering, 2 – Understanding, 3 – Applying, 4 – Analyzing, 5 – Evaluating, and 6 – Creating. Average use of these levels could show a general progression from simple remembering (e.g. defining, telling, listing) to application (identifying, selecting, organizing) to creating (imagining, elaborating, solving). Once the essays were coded, interrater reliability was determined by using the intra-class correlation coefficient function of SPSS v. 20 to determine a kappa score of reliability, with a score of .80 deemed reliable.

Results, the observed pattern, allowed me to see a progress toward more complex reasoning in the assignment as the class progressed and students gathered more information and became more comfortable with theory application. Briefly stated, the first essays indicated an average response at Bloom’s applying level. Students were identifying and correctly applying concrete elements of the theories and making tentative, but informed, inferences. However, by the final essay the average response level was solidly at the evaluating level. There, students were appraising the flexibility of the theories being applied along with the documentary participant they were following. It became more common to see students suggest multiple possibilities in their writing, prioritize these, and determine the most informed interpretation. Consequently, pattern-matching indicated an established theoretical progression in reflective thinking from pre- to post-test.

Still, very few specific examples of best-practice exists with pattern-matching (Almutairi et al, 2014) and applications in SoTL (and education, in general) are rare. However, there are a number of available theories that could be considered as an identified pattern. For instance, I am currently using William Perry’s (1999) scheme of intellectual develop during the college years as a pattern basis in order to better understand contemporary student’s willingness, or unwillingness, to discuss racism in the classroom. Perry’s scheme is noticeably related to Bloom’s work, though somewhat better suited to assess student readiness to learn. Lastly, there are several other established variations of pattern-matching. In you are interested, a good place to begin would be Robert K. Yin’s (2009), Case study research: Design and methods.

References

Almutairi, A.F., Gardner, G.E., & McCarthy, A. (2014). Practical guidance for the use of pattern-matching technique in case-study research: A case presentation. Nursing and Health Sciences, 16, 239-244. doi: 10.1111/nhs.12096.

Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. London: Longman, Inc.

Perry, W. (1999). Forms of ethical and intellectual development in the college years: A scheme. San Francisco: Jossey-Bass.

Yin, R.K. (1994). Case study research: Design and methods (2nd Ed.). London: Sage.

 

 

 

Advertisements


Leave a comment

Making the Case for SoTL Self-Advocacy in Academic Job Searches

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor, Communication Sciences and Disorders at Illinois State University

Two close friends are currently on the job market. Presently employed as associate professors (or close to that rank!) each has chosen to seek novel adventures elsewhere in academia. Thus, they are in the process of sifting through position descriptions and polishing their teaching and research statements to submit in the coming weeks, following a course familiar to many in higher education.

search1Each of my friends have been successful disciplinary researchers as well as productive SoTL scholars, though they both represent disciplines that do not consistently value SoTL. As they contemplate phone interviews and campus visits in the near future, they have wondered aloud about how they might contextualize their SoTL work in a way that positions them well in their job searches. While it’s disappointing to me that they have to consider this issue (SoTL should be uniformly valued!), I recognize it’s likely very necessary and, in fact, is smart preparation for their respective job searches.

With that in mind, the contextualization my friends seek as a framework for their SoTL work could actually be a form of SoTL self-advocacy, which I’d define as anything a person does to describe the value of their SoTL work to relevant stakeholders. SoTL self-advocacy might look different across contexts, but in the milieu of a job search, there are definitely steps my friends could take to share the appeal and impact of their SoTL work. Specifically, I would advise each (and would tell others!) that they might do the following to engage in SoTL self-advocacy:

  • Closely examine the mission/vision statements for any institution of interest. Mine these statements for alignment with past and current SoTL projects. Prepare an explanation for how your own SoTL work meshes with or could advance the mission/vision of the institution of interest.
  • Read the strategic plan carefully, noting where your SoTL work matches with current initiatives/efforts being undertaken by your institution of interest. Seek out mentions of student involvement in research, evidence-based instruction, or broad definitions of scholarship. Use these to frame your SoTL work and plans for the future in the interview process.
  • Reflect on the impact of your SoTL work. Many classroom-based SoTL studies lead to changes in curriculum, teaching methods, course design, etc. Be prepared to discuss how SoTL has impacted your teaching or your students’ learning. Have there been impacts to your current department (e.g., curricular changes) that resulted from your SoTL work? These would be important to describe.
  • Closely examine the vitas (hopefully archived online!) of faculty in the department/school/unit you might be looking to join. Determine if any individuals are SoTL-active or could serve as collaborators in the future. Be prepared to talk about how you would engage colleagues in institution-relevant collaborative projects.
  • Visit websites for the teaching and learning center, provost’s office, and/or any other important campus units affiliated with your institution of interest. Look for prompts that support SoTL and precedents for SoTL engagement. Identify individuals on campus who might be in the position to discuss SoTL opportunities at the institution and (if appropriate) attempt to have them included in your on campus interview (or express interest in meeting them if they are not in your interview).

Other ideas for SoTL self-advocacy on the job search? Please post below in the comments section!


Leave a comment

How Are We SoTL-ing?

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University 

In the run-up to ISSoTL 2017 last week in Calgary, Alberta, Canada, it might have been easy to miss that the latest issue of Teaching and Learning Inquiry (TLI), the journal of the International Society of the Scholarship of Teaching and Learning, has just been published. I had the opportunity to read several articles in this issue prior to traveling to the conference and was particularly interested in one article, Survey of Research Approaches Utilised in the Scholarship of Teaching and Learning Publications, which was co-authored by Aysha Divan (U. of Leeds), Lynn Ludwig (U. of Wisconsin-Stevens Point), Kelly Matthews (U. of Queensland), Phillip Motley (Elon U.), and Ana Tomljenovic-Berube (McMaster U.).

Why the interest? As a SoTL faculty/student developer, I am forever asked if there is a “preferred” method for engaging in SoTL. I have always addressed this topic from an anecdotal perspective, simply telling novice SoTL scholars that qualitative, quantitative, and/or mixed methods are all equally appropriate for SoTL, depending on the “fit” of the method to the study aims/design. With this paper, a bit more clarity was offered as a result of systematic study of three years of published SoTL journal articles.

Honestly, I imagined that there were far more qualitative methods employed in SoTL research than quantitative; however, I was incorrect. Overall, 223 articles from the following journals were studied: International Journal for the Scholarship of Teaching and Learning, the International Journal for Teaching and Learning in Higher Education, and the Journal of the Scholarship of Teaching and Learning. Across these articles, there was an almost even balance of quantitative, qualitative, and mixed methods research (see graphic below).

methods breakdown

Of even greater surprise to me were the following findings:

  • 84% of papers utilized a single data source for reporting (primarily students), which leaves the need for triangulation of data open for consideration in terms of future project planning.
  • Data from mixed methods studies were often times poorly integrated with only 30% of studies fully integrating qualitative and quantitative data as part of the discussion of findings.
  • 65% of studies relied on a single “snapshot” of data (data collected at one time only), which leads to thoughts on the value of/need for collecting longitudinal data to study student learning over time.

At ISSoTL last week, Gary Poole delivered a plenary address reminding us all that as professionals interested in SoTL, we have a choice to facilitate or hinder as we collaborate and mentor. As a professional developer for faculty and students interested in SoTL, I intend to share this information as a facilitative effort to grow SoTL at ISU (and beyond), helping future SoTL scholars to be mindful of trends, needs, and considerations in SoTL publishing. Specifically, I will urge SoTL researchers to:

  1. Seek out a “goldilocks” fit to connect their research questions to the type of data they collect. Why? This allows a researcher to determine whether research question(s) being posed are best answerable with qualitative, quantitative, or mixed methods approaches. A good fit is critical for a study to make sense to interested stakeholders.
  2. Ensure that data come from as many direct data sources as are necessary to form a strong foundation for any discussion of results/implications.
  3. Use indirect data sources primarily as support/triangulation for data collected from direct sources.
  4. Think carefully and critically about how data from a study is discussed. If the design selected has a mixed methods approach to data collection, then all aspects of data should be explored in an integrated manner to identify trends and accurately interpret and report data across the board.
  5. Consider whether data collected at multiple data points might be more appropriate for a study than a “one-time” data collection effort in order to best answer the research question(s) being posed.

 

Blog References:

Divan, A., Ludwig, L. O., Matthews, K. E., Motley, P. M., & Tomljenovic-Berube, A. M. (2017). Survey of research approaches utilised in the scholarship of teaching and learning publications. Teaching and Learning Inquiry, 5(2).


Leave a comment

Advocacy & Outreach Sessions at ISSoTL in Calgary

Compiled by Jennifer Friberg, Cross Endowed Chair in SoTL at Illinois State University and co-chair of ISSoTL’s Advocacy & Outreach committee

Next week, SoTL folks from all over the world will gather in Calgary, Alberta, Canada for the 14th annual conference of the International Society for the Scholarship of Teaching and Learning (ISSoTL). For this conference, ISSoTL’s Advocacy & Outreach (A&O) committee has developed three panels to discuss needs and opportunities to support SoTL locally and globally. Please join our committee as we facilitate the following panels:

Addressing Issues of Our Times

  • Thursday, 10/12/17 from 8:30-10am – Glen 201
  • Panelists: Lauren Scharff (U.S. Air Force Academy), Jennifer Friberg (Illinois State University), Allison Meder (University of Kansas), Clair Hamshire (Manchester Metropolitan University), and Arshad Ahmad (McMaster University)
  • This panel will share perspectives and lead discussion centered on how we (individual ISSoTL members, the A&O committee, and/or ISSoTL at large) might engage in and support appropriate responses to local, state, national, and international issues that relate to or affect SoTL.

Teaching Stream Positions: Mapping and Advocating for SoTL in Diverse Landscapes

  • Thursday, 10/12/18 from 4-5:30pm – Glen 209
  • Panelists: Diana Gregory (Kennesaw State University), Arshad, Ahmad (McMaster University), Mary Huber (Carnegie Foundation), Trent Maurer (Georgia Southern University), Nicola Simmons (Brock University)
  • The panel will explore the diverse landscapes of teaching stream positions from various institutional perspectives while examining the role of SoTL in how various teaching positions are defined, supported, and evaluated.

Social Media Strategies for SoTL

  • Saturday, 10/14/18 from 8-9:30am – Glen 203
  • Workshop Facilitators: Raj Chaudhury (University of South Alabama), Sophia Abbot (Trinity University), Phillip Edwards (University of North Carolina, Chapel Hill), John Draeger (Buffalo State University), Jennifer Friberg (Illinois State University)
  • This panel will provide a guided, practical approach to assist either individuals or institutional units that aim to be more intentional in their social media outreach to champion SoTL. This workshop will focus on four specific social media platforms: Twitter, Facebook, blogs, and YouTube.