The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Photo Documentation: SoTL Methods Series #4

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

Over the last five years, students from my academic department have traveled to six countries on two continents as part of our program’s study abroad experience. Offered for independent study credit, these study abroad experiences have transpired as “short term” opportunities (e.g., spring break or two weeks in the summer session) and have focused on cultural immersion, rather than disciplinary content knowledge. I led our most recent trip to Spain during spring break 2017 and was joined by two faculty colleagues and 32 students. Prior to this year’s trip, students self-reported gains in what Miller-Perrin and Thompson (2014) would consider internal learning (e.g., emotional, intellectual, and spiritual growth) as well as external learning (e.g., second language acquisition, intercultural learning, globalization, disciplinary knowledge). While this was lovely to hear, these impressions were anecdotal, as we had never systematically studied student learning that occurred as a result of engagement as a study abroad participant.

I decided to study this year’s students to measure the impact of short term study abroad for this cohort. Initially, I thought I would restrict my study design to traditional methods of reflection and content analysis. In the weeks prior to travel, I asked students to set three goals for themselves based on self-perceived areas of weakness or interest. Students were asked to keep track of their growth in these goal areas via reflective journals, which were analyzed carefully for evidence of progress or growth in their goal areas. Eighty-one of these goals fit into one of the following categories: taking chances, engagement, flexibility, gaining independence, archiving, budgeting, social interaction, or understanding culture.

Around the time that I was developing the plan for collecting data from travelers, I read about photo documentation, a method derived from visual sociology wherein researchers seek out patterns in photographic data collected using something called a shooting script. I was intrigued as to how I might introduce this new method into my study abroad project. I tried, and I was able to extend what I learned about my students in doing so. The remainder of this post is a description of my “rookie experience” with photo documentation. I am certainly not an expert, but I did enjoy exploring this visual methodology!

Photo documentation is a research method developed by Charles Suchar (1997). Specifically, “photo documentation is a method that assumes photographs are accurate records of what was in front of the camera when its shutter snapped – ‘a precise record of material reality’ — and takes photographs in a systematic way in order to provide data which the researcher then analyzes” (Rose, 2016, p. 310). The key to photo documentation is a shooting script which consists of “lists of sub-questions” (derived as topical to overarching research questions) which act as a guide for taking pictures connected to the topic of the research being conducted (Rose, 2016, p. 311). Photos are taken in accordance with a shooting script, then analyzed for categories and patterns via a systematic coding process.

In the case of the study I described above, I wanted to understand changes that occurred in students in self-identified areas of need/interest. Thus, in addition to reflective (written) journaling, I asked that students use the following shooting script to document learning visually before and during travel:

  1. What was I doing/seeing when I recognized that I had made progress toward meeting one of my goals?
  2. What things/people/experiences influenced this progress?

spain food 2In applying this shooting script, students were directed to take pictures to answer these questions as they were going about their study abroad experiences. After travel, they submitted three photos per goal to me, along with their written journals. I printed out all photos that students submitted and searched for patterns across goals and photos. I was able to find many! As an example, many students set goals to improve their understanding of different cultures. Looking at the photos submitted along with written reflection for this goal, I could see that students represented “increases in cultural understanding”  via photos of food (see right), religious symbols, architecture, or Spanish citizens they interacted with during their travels. Students who set goals to “take chances” maggierepresented growth in this area with photos of new foods they tried or photos of experiences where they conquered personal fears (e.g., communing with the macaques in Gibraltar after a prior bad experience — see left).

Overall, what I found interesting is the different stories these data told. While written reflections told me WHAT changes students had realized as a result of study abroad, the photographs told me HOW how these changes occurred. The combination of these different data types were powerful to tell the story of my students’ experiences in Spain and made a powerful case for the possibility of significant learning in a short amount of time.

I have always been intrigued with what the visual representation of learning might look like for different students. Using photo documentation helped me to see the possibilities of visually-based data in a way that I appreciate and hope to use again in the future!

Blog References:

Miller-Perrin, C. & Thompson, D. (2014). Outcomes of global education: External and internal change   associated with study abroad. New Directions for Student Services, 146, p. 77-89.

Rose, G. (2016). Visual methodologies: An introduction to researching with visual methods (4th ed.). Sage: Los Angeles.

Suchar, C. S. (1997). Grounding visual sociology research in shooting scripts. Qualitative Sociology, 20(1), 33-55.

 

Advertisements


1 Comment

Think Alouds: SoTL Methods Series #3

Written by Sarah M. Ginsberg, Ed.D., Professor of Speech-Language Pathology at Eastern Michigan University (sginsberg@emich.edu)

Editor’s Note: This blog was originally posted on the SoTL Advocate on October 12. 2015 and is reprinted in its entirety now for its excellent fit in the current methods series which features methods for SoTL that are “new and different” to many.  — JCF

Screen Shot 2017-11-13 at 8.25.05 PMA common thread that runs through various cross-disciplinary SOTL research is the concerted effort made to understand what the accomplished professional is thinking when she solves a work problem so that we can use that knowledge as teachers to better prepare future professionals. That problem might be how a mathematician completes a technical calculation, or in clinical fields, it might be how the clinician arrives at a diagnosis. The value for all of us in understanding what our accomplished colleagues do in their heads when faced with a technical problem is that in identifying how the pros do it, we can uncover insights into how we should be teaching our students to think and to problem solve. This type of understanding relies on a process of collecting data while the person is actively engaged in solving a problem out loud. This type of study is often referred to as a think aloud (TA) and can yield important information to inform evidence-based educational practices

The TA method is a validated method of learning about cognitive processes by having participants verbalize their thinking in a metacognitive manner (Ericcson & Simon, 1993; Wineberg, 1991). TAs were popularized by Wineberg (1991) in his ground-breaking study that examined the differences between how academic historians processed information while reading historical texts and how students processed information regarding historical texts.  Since then, TAs have been used to study how novice thinking compares to experienced thinking in a wide variety of disciplines, including the health sciences, mathematics, and political science (Banning, 2008; Bernstein, 2010; Forsberg, Ziegert, Hult, & Fors, 2013; Wainwright & McGinnis, 2009). These types of studies are often referred to as “expert-novice” studies (Bernstein, 2010).

The process of data collection using a TA approach is quite simple and requires minimal technology and cost. Typically:

  1. Study participants are presented with the problem to be solved by the researcher and asked to solve it aloud.
  2. Specific directions are provided to participants. Prompts (e.g., “tell me how you would solve this” or “describe how you would approach this problem”) are used to elicit responses and gather additional information if a participant falls silent or struggles with the process.
  3. Participant responses are recorded for subsequent transcription and analysis.
  4. Once the TA is transcribed, the most challenging part of the process becomes the subsequent data analysis. Consistent with qualitative methodology, verbalizations may be read as a whole to determine initial emerging codes and impressions about the thought process (Creswell, 2002; Denzin & Lincoln, 2012). Using an inductive approach to identifying specific thought processes or strategies allows the researcher to move forward to developing secondary, axial coding. Themes emerge as the iterative process expands to include all participants and commonalities and differences can be appreciated.

Having recently completed a study comparing the diagnostic problem-solving of experienced speech-language pathology (SLP) clinicians compared to the problem-solving of SLP graduate students, I found that the most challenging aspect of analyzing the data was determining the level of thinking to focus on. I used studies in comparable clinical professions, such as nursing, physicians, and physical therapists to identify frameworks that might be useful to me. In determining the focus of my study, I chose to concentrate on the heuristics (thinking strategies) of my participants, to understand differences in approaches to problem solving and to create a framework that fostered comparisons to previous literature, potentially increasing the value of my findings.

For more details on the think aloud method and some outstanding examples of its use in a variety of fields, see the items included in the following references. It should be noted that a number of authors also advocate for the use of TA as a teaching method. For those unfamiliar with qualitative research methodology, several references are included here as well.

References for Additional Information on Think Alouds:

Banning, M. (2008b). The think aloud approach as an educational tool to develop and assess clinical reasoning in undergraduate students. Nurse Education Today, 28, (1), p. 8–14. doi: 10.1016/j.nedt.2007.02.001

Bernstein, J. L. (2010). Using “think-alouds” to understand variations in political thinking. Journal of Political Science Education, 6(1), p 49-69. doi:10.1080/15512160903467695

Ericcson, K. A., & Simon, H A. (1993). Protocol analysis: Verbal reports as data. Cambridge, MA: MIT Press.

Forsberg, E., Ziegert, K., Hult, H., & Fors, U. (2013). Clinical reasoning in nursing, a think-aloud study using virtual patients-A base for innovative assessment. Nurse Education Today, http://dx.doi.org./10.1016/j.nedt.2013.07.010

Wainwright, S. F., & McGinnis, P.Q. (2009). Factors that influence the clinical decision-making of rehabilitation professionals in long-term care settings. Journal of Allied Health, 38(3), 143-51.

Wineberg, S. S. (1991) On the reading of historical texts: Notes on the breach between school and academy. American Educational Research Journal, 28(3), 495-519.


Leave a comment

#Collaborative Auto-ethnography: SoTL Methods Series #2

Written by: Catherine McConnell (University of Brighton), Elizabeth Marquis (McMaster University), and Lucy Mercer-Mapstone (University of Queensland) — note complete author-supplied affiliations and contact information at the end of this blog post. 

When we (Catherine, Beth, and Lucy) met at the International Summer Institute on Students as Partners in 2016 we quickly discovered that, as practitioners of student-staff partnerships, we had many shared experiences. Our discussions were fruitful in terms of giving us a sense of belonging but we felt they warranted deeper exploration. So, we embarked on a process of delving into our own experiences in the hopes of learning in the process and sharing that learning with fellow practitioners and researchers. As we worked to find a way to effectively and systematically study ourselves, we arrived at the idea of using collaborative auto-ethnography as a methodological approach.


Above: Participants at the 2016 McMaster Summer Institute on Students as Partners in Teaching and Learning, where we first developed the idea for our CAE project.


Collaborative auto-ethnography (CAE) is a research method that involves ‘a group of researchers pooling their stories to find some commonalities and differences […] to discover the meanings of the stories in relation to their sociocultural contexts’ (Chang et al, 2013, p. 17). As such, it provided a perfect, if initially somewhat unfamiliar, way to collectively explore our individual experiences in a scholarly fashion.

We have put together this blog post to explain our developing understanding of the method and process of CAE, and how we, as a group of three researchers, have used it in our recent SoTL enquiry into ‘student-staff partnership in higher education.’

CAE builds upon ‘auto-ethnography,’ which is a method that uses a researcher’s personal experience to ‘describe and critique cultural beliefs, practices, and experiences’ (Adams et al, 2015, p.1). Auto-ethnography is a deeply personal and reflective process, usually taking place in a researcher’s own context, whether that be about practice or in a certain situation. Adding the dimension of multiple ethnographies (i.e., more than one voice) presents the possibility that the method can simultaneously be collaborative, autobiographical, and ethnographic.

For our shared research, we took up the CAE method through structured reflective writing, designing a writing activity to provide a framework that would guide our individual reflections. We set ourselves a word limit of 750 words that we would write and share with each other in a private online space. This activity was modelled on a set of reflective prompts, informed by Johns’ model of structured reflection (Johns, 2000), which had been adapted by colleagues at the University of Brighton, UK (2011) and which we subsequently re-appropriated. Specifically, we:

  1. Developed a framework of prompts/questions for reflection
  2. Wrote individual reflections guided by the framework
  3. Shared and read each other’s reflections
  4. Conducted iterative thematic analysis to discover key themes

Our developed framework was a useful facilitative tool to prompt us to take an open, honest account of our personal and professional experiences, affirming Johns’ interest in ‘making explicit the knowledge we use in our practice’ (Jasper, 2013, p.86). Posing questions that follow Johns’ (ibid, p.37) format of phased cue questions (phases involve preparatory, descriptive, reflective, anticipatory, and insight questions) we focused our SoTL enquiry on identity construction, navigation and enactment in the context of student-staff partnerships.

The framework below provides an illustration of the types of questions used to prompt our personal reflections about identity in the context of student-staff partnerships. This could be easily adapted to other SoTL topics –  especially those that reflect on practice.

Framework for reflective questioning
Description of experience Phenomenon: describe in detail your partnership practice, or a specific partnership experience that seems especially noteworthy, without interpretation or judgement
Context: what were the significant background factors to this experience? Why did it take place, and what was its purpose?
Reflection What were you trying to achieve?
Why did you behave as you did?
What were the consequences of your actions for yourself and others?
How did you feel about the experience when it was happening?
To what extent did your actions realize your understanding of partnership?
What identity(s) were you consciously aware of at the time?
What identity(s) do you believe were at play during this interaction in hindsight?
If multiple, which identity was most salient? How were they interacting?
Analysis: Influencing factors

 

What factors influenced your decision-making? Some potential options to consider: Prior experiences, Societal expectations/ideologies/assumptions, Context
How was your salient identity affecting your actions?
How was the interaction between identities affecting your actions?
How was the presence of this identity(s) influencing your perceptions of those with whom you were interacting?
Analysis: Alternatives What other choices did you have?
What could be the consequences of these choices?
Learning & Action How do you feel about this experience now?
Has this experience changed your way of understanding yourself?
Did your salient identity change? If so, how and why?
In hindsight, how has is interaction/even affected your ongoing identity in partnership?
What new questions, challenges or issues has it raised?
Given the chance, what would you do differently next time?
How will you follow up on this experience in order to put your learning into practice?

Once we had written and shared our individual reflections, we found it useful to read each other’s, and write a short ‘meta-reflection’ on the writing process. This enabled us to appraise the CAE method straight after the reflective process but before we began any analysis of the transcripts. Some of our observations included:

  • The researchers felt a sense of belonging and solidarity to one another along with a communal ownership of an enquiry
  • Writing and sharing reflections caused each of us a personal realisation and provided grounds for transformation. This process was not without some discomfort, though, perhaps because there is a level of vulnerable-making involved with sharing personal reflections with colleagues
  • Sometimes the content of the reflections themselves was also unsettling or challenging, and caused discomfort in relation to an aspect of one’s own practice
  • We experienced a heightened consciousness of our own values and beliefs relating to practice, the influence these have in partnership situations, and in our expectations of others
  • We also noted a sense of excitement when reading each other’s writing inspired by the experience of sharing personal insights and aspects of our own identity that are usually implicit

While CAE proved a useful method for meaningfully exploring our research questions about our own experiences, then, it was also an exciting and sometimes uncomfortable process that supported reflective thinking and potential development of our practice as teachers and learners.

Project Information

We are three SoTL practitioners working in the area of student-staff partnership in HE across three western countries, in differing roles, and in significantly different institutional contexts. Catherine McConnell is a Senior Lecturer in a Learning and Teaching centre, focusing her work and doctoral research on student-staff partnership, at the University of Brighton in the UK. Beth Marquis is Associate Director (Research) at the central teaching and learning institute at McMaster University in Canada. Her disciplinary training is in film and cultural studies. Lucy Mercer-Mapstone is a PhD candidate and research co-fellow at the University of Queensland, Australia. She currently leads the collaborative design of a university-wide Students as Partners program that aims to embed a culture of partnership at the institution.

You can find out more about our study: Breaking Tradition Through Partnership: Navigating Identities and Dissonance in Student-Staff Partnerships in the EuroSoTL Conference Proceedings, p296.

Catherine McConnell * a, Elizabeth Marquis b, c Lucy Mercer-Mapstone

a Centre for Learning and Teaching, University of Brighton, Sussex, England. C.McConnell@brighton.ac.uk

b Arts & Science Program and MacPherson Institute, McMaster University. Hamilton, ON, Canada. beth.marquis@mcmaster.ca

c Institute for Teaching and Learning Innovation, University of Queensland, St Lucia, Brisbane, 4072, Australia, l.mercermapstone@uq.edu.au, orcid.org/0000-0001-7441-6568

*Corresponding author

 

Blog References

Adams, T. E., Linn, H. J. S. and Ellis, C. (2015). Autoethnography. Oxford: Oxford University Press.

Chang, H., Ngunjiri, F., & Hernandez, W. (2013). Collaborative autoethnography. London: Routledge.

Jasper, M. (2013) Beginning Reflective Practice. (2nd edition) Hampshire: Cengage Learning.

Johns, C. (2000) Becoming a Reflective Practitioner. Oxford: Blackwell Scientific Publications.

University of Brighton. (2011). Critical Incident Analyses. Brighton: University of Brighton. 


Leave a comment

Theoretical Pattern-Matching in SoTL: SoTL Methods Series #1

This blog serves as the beginning of a four-week focus on unique research methods for SoTL work. Enjoy, and please feel free to write to our guest bloggers with any feedback or questions! -Jen Friberg, blog editor

Written by: Bill Anderson, Associate Professor of Family and Consumer Sciences at Illinois State University (jander2@ilstu.edu)

I recently completed a SoTL project where I was hoping to create vicarious, but meaningful, applications of classroom learning, in this case, foundational theories of the human development field. In an attempt to accomplish this, I utilized interrupted case studies (ICS), a progressive disclosure of information viewed as problem-based learning over time. Over an eight week period following a pre-test application, students viewed a longitudinal series of interviews as an ICS. This series followed several participants from the time they were seven years old in 1964, revisiting them every seven years until age 56 in 2013. During the process, and using the assumptions, concepts, and language of assigned developmental theorists, students described and applied relevant theoretical positions to anticipate growth and change as this collection of real lives progressed. Their work was submitted in weekly reflective essays. At the end of the eight-week assignment, post-test results indicated that the method was quite successful but told me nothing further. The post-test increase could simply be the result of memorizing the material. Therefore, pattern-matching was applied to further examine those results.

patternPattern-matching is a less-known, but dependable, procedure for theory testing with case-studies and is regularly recommended for reconciling mixed methods and data sources in case study research, and to boost the rigor of the study. The overarching goal is to explain relationships between key points, in this case the pre/post results, by comparing an identified theoretical pattern with an observed pattern.

The previously mentioned weekly student essays were utilized as the observed pattern. These included descriptions of their assigned interview participants, appraisals of their most recent developmental predictions for this person, and their expectations for the next seven years. The essays were coded line-by-line to determine the degree of matching to a predetermined theoretical pattern. In this case, Bloom’s Revised Taxonomy (Anderson & Krathwohl, 2001) was utilized as coding categories as follows: 1 – Remembering, 2 – Understanding, 3 – Applying, 4 – Analyzing, 5 – Evaluating, and 6 – Creating. Average use of these levels could show a general progression from simple remembering (e.g. defining, telling, listing) to application (identifying, selecting, organizing) to creating (imagining, elaborating, solving). Once the essays were coded, interrater reliability was determined by using the intra-class correlation coefficient function of SPSS v. 20 to determine a kappa score of reliability, with a score of .80 deemed reliable.

Results, the observed pattern, allowed me to see a progress toward more complex reasoning in the assignment as the class progressed and students gathered more information and became more comfortable with theory application. Briefly stated, the first essays indicated an average response at Bloom’s applying level. Students were identifying and correctly applying concrete elements of the theories and making tentative, but informed, inferences. However, by the final essay the average response level was solidly at the evaluating level. There, students were appraising the flexibility of the theories being applied along with the documentary participant they were following. It became more common to see students suggest multiple possibilities in their writing, prioritize these, and determine the most informed interpretation. Consequently, pattern-matching indicated an established theoretical progression in reflective thinking from pre- to post-test.

Still, very few specific examples of best-practice exists with pattern-matching (Almutairi et al, 2014) and applications in SoTL (and education, in general) are rare. However, there are a number of available theories that could be considered as an identified pattern. For instance, I am currently using William Perry’s (1999) scheme of intellectual develop during the college years as a pattern basis in order to better understand contemporary student’s willingness, or unwillingness, to discuss racism in the classroom. Perry’s scheme is noticeably related to Bloom’s work, though somewhat better suited to assess student readiness to learn. Lastly, there are several other established variations of pattern-matching. In you are interested, a good place to begin would be Robert K. Yin’s (2009), Case study research: Design and methods.

References

Almutairi, A.F., Gardner, G.E., & McCarthy, A. (2014). Practical guidance for the use of pattern-matching technique in case-study research: A case presentation. Nursing and Health Sciences, 16, 239-244. doi: 10.1111/nhs.12096.

Anderson, L.W., & Krathwohl, D.R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Blooms taxonomy of educational objectives. London: Longman, Inc.

Perry, W. (1999). Forms of ethical and intellectual development in the college years: A scheme. San Francisco: Jossey-Bass.

Yin, R.K. (1994). Case study research: Design and methods (2nd Ed.). London: Sage.

 

 

 


1 Comment

Seeking Blog Contributors for Fall 2017 SoTL Methods Series

Written by: Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

searchIn the early days of the SoTL Advocate, I featured a short-lived methods blog series that provided overviews of three research methods common to SoTL inquiry: case study, content analysis, and survey. These blogs functioned as brief overviews of each method and provided readers with resources to better understand these methods and exemplar articles to access, as well. Recently, several readers have asked for this series to be expanded, which I think is a wonderful idea! To that end, this fall, I plan to offer a multi-week continuation of the SoTL methods series, with specific methods included in this series to be determined.

Why are the topics yet to be determined? I hope to feature guest blog contributors in this series to represent the interesting and broad approaches to SoTL across disciplines and countries. It is my aim that each submitted blog will:

  1. Define/describe the method of focus for the blog.
  2. Provide an overview of a project where this method was used, along with a reflection on WHY this method was selected over others.
  3. Offer resources for readers to view other examples or descriptions of this method in SoTL (preferred) or discipline-specific scholarship.
  4. Cite references for all resources noted in the blog.
  5. Provide affiliations and contact information for all blog contributors.

Do not feel as though you have to be a recognized “expert” on the method you write about — you just have to be willing to share what you’ve learned through reading or using the method you have chosen. Single author contributions from students or faculty are welcome, but please feel free to invite colleagues and/or students to co-contribute, as well.

Blogs should be approximately 750 words in length and should be written in a friendly and accessible manner, absent unneeded disciplinary jargon that might make a general SoTL readership unable to benefit from accessing the content of the post. Visuals (e.g., open source pictures, photos, videos) are encouraged, as more people will “click” on a blog link if a visual is attached!

If you are interested in submitting a blog for this series, please email me, Jen Friberg (jfribe@ilstu.edu), with a brief statement of interest by August 1, 2017 as I want to ensure we do not have unnecessary overlap in topics. Final blogs should submitted to me by September 15, 2017 for review and formatting. It is anticipated that this methods series will be featured in the SoTL Advocate from October-November, 2017.

A bit of information about the SoTL Advocate blog (i.e., history and reach) is presented below:

About the Blog: The SoTL Advocate was established by the Office of the Cross Endowed Chair in the Scholarship of Teaching and Learning at Illinois State University (ISU) to highlight interesting SoTL work and encourage discussion within the SoTL community on various topics of interest to those working on SoTL at ISU and beyond. It is the goal of the SoTL Advocate that blogs will feature viewpoints of a diverse authorship, discussing SoTL projects, reflections, ideas, and topics that are representative of the global nature of the study of teaching and learning.

Blog Reach: Since November 2014, over 7000 visitors (representing 56 countries) have viewed blog content. On average, the SoTL Advocate is accessed over 40 times a week by unique viewers. All blog posts are publicized via the Twitter (250 followers) and Facebook (75 followers) accounts managed by the Office of the Cross Endowed Chair in SoTL. Blog authors can request specific hashtags/attributions for these posts, as appropriate.

Blog Post Guidelines: Blogs should be approximately 750 words (500-1000 word range is acceptable). Blogs should be written in a friendly and accessible manner, absent unneeded disciplinary jargon that might make a general SoTL readership unable to benefit from accessing the content of the post. Visuals (e.g., open source pictures, photos, videos) are encouraged, as more people will “click” on a blog link if a visual is attached!

Submission of a blog does not guarantee acceptance for publication. All blog submissions are reviewed by the SoTL Advocate editor for content and form prior to notification of acceptance status. Please note that blog posts may be conditionally accepted for publication pending revision/clarification. Blogs accepted for publication under this call for contributors will be published between October and November of 2017 as part of the SoTL Methods Series.