The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Bridging the Divide between Content and Pedagogy: Reflective History Teaching

Written by: Richard Hughes, Associate Professor of History at Illinois State University and Sarah Drake Brown, Associate Professor of History at Ball State University

A common feature of universities is the dual role of addressing disciplinary content and, either directly or indirectly, preparing future generations of students to teach the discipline. A product of a research project entitled, “Historians and History Teachers: Collaborative Conversations,” this post summarizes our presentation at the Illinois State University Teaching and Learning Symposium in January2016 and explores our effort to address the persistent divide between content and pedagogy among undergraduate history education students. History educators often debate the number of required history courses in relation to education courses for undergraduates seeking to be teachers. To address this issue, we created a course, “United States in the Twentieth Century,” that purposefully blended historical content, the burgeoning scholarship in history education, and reflective practice. We drew upon SoTL work in history and scholarship in history education as the basis for a course designed to build teaching candidates’ pedagogical content knowledge and prepare them for content methods courses. Our work explored the following question: “How do students’ disciplinary understandings affect their emerging conceptualization of discipline-specific teaching?”

The research project included a number of assignments aimed to assess teaching candidates’ understanding of disciplinary concepts and how these understandings contribute to the development of pedagogical content knowledge in history. Some of the specific areas included how future teachers understand historical knowledge and the work of historians and how they craft historical narratives for their students. The 17 students who participated in the study were all junior and senior history education majors with, on average, eight or nine previous college courses in history including our department’s required course in historical methods and a grade point average of 3.0 or higher. In other words, these history majors were relatively accomplished, experienced, and motivated. They will also make innumerable decisions in history classrooms in the future.

Two of the assessments in the course involved questions about the role of individuals involved in history education and “think aloud” interviews in which participants were also researchers in history education. The first assessment involved asking ISU history education students three simple questions: “What do historians do?” “What do history teachers do?” and “What do history students do?” The following word clouds represent the answers to each question:

“What do historians do?”

hughes 1

“What do history teachers do?”

 hughes 2

“What do history students do?”

hughes 3

The differences in the students’ answers for each question were substantial. For the first question, students’ answers were full of action verbs that conveyed how historians engage the past and create knowledge as part of a process. Their answers suggested that students were very clear that history does not simply equal the past but rather represents how historians construct the past with evidence. In contrast, students’ answers about teaching were surprisingly vague about what teachers do other than “teach.” Their written answers suggested a limited role in which teachers are simply passive vehicles for distributing facts as if history was an object that teachers transferred to their students. A few examples were more revealing than the word cloud:

  1. “History teachers are responsible for passing along the information found by historians and guiding them to think historically.”
  2. “History teachers use the information gathered by historians (or themselves) and use it to describe events and characters to students.”

The students’ answers to the final question about what history students do were even less specific and often limited to the verb “learn.” When students were more specific, they most often mentioned assignments or skills such as researching, writing, or taking notes. No students made any explicit reference to historical thinking, and many of the answers about the role of history students were so general that they could easily be applied to students in any academic subject. Scholarship in history education has exploded in recent decades with claims about the discipline-specific nature of historical cognition and the importance of teaching historical thinking in secondary and higher education. However, the teaching candidate’s assumptions about the role of historians, teachers, and students provide some significant obstacles to this goal. How likely is it that teachers promote and teach historical thinking among their students if they do not even conceptualize the role of a teacher as involving active and critical engagement with the past?

Another assessment involved asking ISU students to create and conduct a research study as part of our unit on the United States during the Great Depression. After reading and discussing a number of historical issues related to the topic, students selected, analyzed through a “think-aloud,” and recorded their analysis of two important primary sources (one text, one image) from the period. They were then paired up with a freshman from University High School and recorded the high school students as they analyzed (out loud) the sources. Each participant also conducted the short experiment with an ISU student who was not a history major, including a recorded analysis of the same documents. Students then wrote a paper comparing and contrasting the three analyses in light of the research they had read in both U.S. history and history education such as a chapter in Sam Wineburg’s Historical Thinking and Other Unnatural Acts: Charting the Future of Teaching the Past (2001). Here are a few representative examples from the participants’ reflections on the project:

“This experience has truly deepened my understanding of historical thinking, not only by reminding me what it means to think historically myself, but also to consider how others may think when reading sources and documents, which is extremely pertinent to all who value the importance of understanding the past and future.”

“It is important to note that many of the students that are being interviewed for the project have never been formally introduced to the characteristics of historical thinking, but I believe that this is what makes the project so relevant. By analyzing the students’ thought process[es] you can see the difference between what methods students innately possess as opposed to the methods that we are taught as history majors.”

“This gave me a much better view of how students learn and how students evolve. While it can be hard to remember what it was like to be thirteen or fourteen, doing activities like this can give an important insight into how that age group thinks so we can better teach them.”

“The research experience will inform my future teaching methods and implementation of primary sources in the classroom. I hope to make the distinction between history as an account and history as an event with my students in conjunction with thinking historically. Both methods will be improved through the research as it helped me observe the thinking processes of two different students at two different levels of education.”

The students’ reflections on the project suggested that the experience was powerful and effective in making “visible” (or at least audible) the important cognitive processes that remain hidden but necessary in most college and secondary history classrooms. While all the students reflected on the performance of the secondary and college students, the participants’ essays suggested that many remained largely unaware of their own thinking when engaging the primary sources associated with the 1930s or their crucial role, as emerging teachers, in selecting sources that were or were not conducive to illustrating historical thinking.

Our findings suggest the value of revising curricula across campus to provide future teachers, regardless of subject, with a better understanding of what occurs when individuals engage their discipline. Colleges and universities approach teacher education very differently across the nation. At Illinois State University, we have long taken pride that history teachers are trained by historians within the department of history. The same pattern holds true for secondary teachers in areas such as biology, English, and math. In our department, we are also proud that history education majors take the same required courses in history and complete the same senior capstone assignments as students whose future plans involve a Ph.D. in history, law school, or employment in a history museum.

However, these two experiences with History 309, while only a portion of the research integrated into the class, illustrate that, despite the fact that the university does a great deal right in terms of academic structure, future history teachers perceive teaching history as strangely separate from their experiences as students within the discipline. Despite the fact that history education students at Illinois State already experience two of the recommendations that proponents of history education promote—an academic major in history and a substantial number of history courses—many of them struggle to become the type of teacher consistent with the research in history education. If this is accurate, future history teachers need different kind of history courses rather than simply more traditional content courses or additional coursework in teaching methods. This study, albeit limited in size and scope, suggests the need for courses that explicitly integrate the methodology of historians and the research in history education into content courses to “make the invisible visible.” For those of us who work directly or indirectly in teacher education, it may be beneficial to engage in a continued examination of our discipline’s unique disciplinary concepts and methodology. We must explore ways in which our teacher education program can bridge the gap between traditional content courses and traditional content methods courses, thereby promoting the development of pedagogical content knowledge.

Advertisements


1 Comment

New SoTL E-journal in Art History

Written by: Virginia B. Spivey, Michelle Millar Fisher, and Renee McGarry on behalf of AHTR. Queries should be addressed to info@arthistorypp.org

AHPP_white

Note: This blog is cross-posted on ISSOTL’s blog.

Last year, Art History Teaching Resources (AHTR), a peer-populated open educational resource, began research and development on Art History Pedagogy and Practice (AHPP), a new online, open-access, and peer-reviewed journal launching in fall 2016. Devoted to the scholarship of teaching and learning in art history (SoTL-AH), and funded by the Samuel H. Kress Foundation, this project developed in response to two key issues: first, the recognition that art historians need opportunities to share rigorous pedagogical research produced in the field; and second, the reality that teaching in the discipline has historically been undervalued in both economic and scholarly terms.

To assess the situation, AHTR conducted a preliminary study, including a field-wide survey performed by the research firm of Randi Korn & Associates that drew over 1200 respondents in higher education, art museums, and other arts-related professions. Simultaneously, AHTR undertook a literature review examining 93 academic publications culled from art history, art and museum education, visual and cultural studies, and digital humanities. The findings, published in AHTR’s October 2015 White Paper, revealed that while art historians in higher education frequently talk about and seek out information related to their teaching, the discipline’s major periodicals and professional conferences give minimal attention to pedagogy.  With this clear mandate for the creation of Art History Pedagogy and Practice, an advisory board was formed, a mission statement crafted, and a partnership established with the Graduate Center at the City University of New York to maintain the e-journal on CUNY’s Digital Commons repository.  A Call For Papers has just been released, and publication of the inaugural issue is slated for October 2016.

The AHPP initiative builds on the success of AHTR as space for the exchange of pedagogical ideas in art history. Founded on dual goals to raise the value of the academic labor of teaching and to provide peer support across ranks of tenured, tenure-track, and contingent instructors, AHTR began as a collaboration between Michelle Millar Fisher and Karen Shelby at Baruch College in 2011. Fisher, then a Graduate Teaching Fellow with a background in museum education, and Shelby, then an Assistant Professor of Art History, organized meetings where colleagues shared teaching materials and experiences. Their popularity suggested potential for a digital forum to connect a wider community of practitioners, and gave rise to the arthistoryteachingresources.org website, which launched publicly in 2013 and has grown rapidly to now average over 800 hits each day.  Since January 2015, when the current 2.0 design debuted, it has received more than 267,000 views from over 91,000 educators in K-12, post-secondary institutions, and art museums, and from academic support staff including reference librarians and curriculum designers. AHTR’s administration has similarly expanded to a collective of art historians, working in different professional settings and ranging in experience from early career scholars to those well established in the field.

A key motivation in founding AHPP has been to reinforce the value, complexity, and rigor of the study of teaching and learning. We want SoTL in art history–and in other disciplines–to be recognized as a robust form of scholarship.  We believe this mission can be successfully championed through the combination of OER and peer-reviewed publication.  As the umbrella platform for AHPP, AHTR will continue to push the boundaries of traditional modes of scholarly communication as an OER that facilitates collaboration and sharing in a forum that requires shorter lead time and lighter peer review.

We are especially interested in questions of labor and value in art history teaching as we, in tandem, assess the sustainability of the scholarly publication model in a digital world.  It is worth noting that a Kress grant in 2014 allowed AHTR to pay scholars small stipends to produce open access lesson plans available on the site, but practitioners contribute blog posts with no compensation and the site’s administrative oversight and operating costs are provided voluntarily by the leadership collective. Perhaps ironically, while collaborating to expand AHTR to include AHPP, it became quite clear that journal management would involve a further commitment of unrecognized, and often unpaid, labor.

As AHTR is not formally affiliated with an institution (partnership with the Graduate Center is beneficial and generous but informal) nor a 501c3, it is able to remain financially and administratively independent from the hierarchies established within academia.  Maintaining this autonomy highlights the privileges and pitfalls of working for free as an academic, especially as more and more academics work outside the tenure track. Marginalized scholars, in particular, are more likely to do this sort of work and less likely to be rewarded for it in their career.

This concern speaks to another, equally important need to open up SoTL in art history–including our own project–to critical eyes and feedback around the intersectionality of race, gender, and discourses of global art history.  In a field where a majority of white women perform much of the labor to teach a disciplinary narrative that continues to favor European art, the questions and ethics of labor in the classroom – and the labor of writing about the classroom – come to the foreground, as does the role of race and experience in shaping course content. We hope to bring these questions, and many more from the wider academic community, to bear on the scholarship of teaching and learning with Art History Pedagogy and Practice.


Leave a comment

Designing SoTL Studies for Out-of-Class Learning: Learning as a Process over Time

Written by Erin Mikulec, SoTL Scholar-Mentor, Associate Professor (Teaching & Learning), and Interim Director of the English Language Institute at Illinois State University

Out-of-class learning offers many opportunities to examine student learning outcomes as the result of participation in various activities, ranging from student organizations, study abroad experiences, field-based or service learning projects associated with courses on campus. In order to support researchers who may be interested in engaging in this kind of SoTL work, this post outlines ways in which such a study might be designed. As a SoTL researcher, I have carried out several studies that have examined out-of-class learning and, in the process, have learned a great deal in regard to designing tasks and activities that serve as data measures. Although my own work has focused on study abroad and student organizations, the suggestions presented here may be applied to a variety of SoTL contexts for out-of-class learning.

As with any study, it is important to specify exactly what it is you want to know. When I first began studying out-of-class learning, my guiding questions were broad and general regarding student learning outcomes. However, in looking at out-of-class learning, one thing I’ve come to understand is the value of differentiating between personal and professional learning outcomes that students experience as a result of participation in such activities. For me as a researcher, this allowed me to examine student growth through different lenses beyond that of academics.  Therefore, when thinking about your guiding questions for a study on out-of-class learning, think about how the project, experience or activity may impact students both professionally and personally. For instance, does the activity in the study develop students’ professional skills through a field-based experience? Which ones and how so? Or perhaps the experience supports students in their development of leadership and communication skills? Further still, does the experience allow for opportunities for personal growth and reflection? Although themes often emerge in data analysis, it can be helpful during the design process to be mindful of these differences in learning outcomes.

When thinking about how to design your out-of-class learning outcomes study, consider your initial data points. We know that good SoTL research considers multiple measures of data collection, thus, the means you use to capture this information at the on-set are important. These data measures can be used to identify where your students are in terms of skills and the potential for growth and development as the project or experiences progress. Some possible tasks and activities to do this include:

  • Surveys, which can have Likert-scale items, open-ended questions, or both,
  • Reflection exercises which help students to consider what they hope to gain from the project or experience, what they already know about a certain issue or topic, or what they are most excited about, apprehensive about, etc.,
  • Online group discussions, based on materials provided by the researcher, such as videos, news or research articles about the topic of the project or experience, or simply questions posed by the researcher,
  • Proposed work plans or schedules of tasks to be completed throughout the project or experience,
  • Multimodal means of capturing students’ starting points, such as photos or videos that students create as part of their own documentation of the journey on which they are about to embark.

In my own out-of-class learning research, I have found that it is equally important to think of this as broadly as possible and more as a starting point rather than simply the “pre” in a pre-post scenario. In this way, using SoTL to assess out-of-class learning is more robust and not limited to a before and after picture of learning outcomes. This allows you as a researcher to examine a progression of learning outcomes and even, as will be discussed next, at which points in the project or experience these outcomes begin to take shape.

Once the project or experience has begun, consider the ways in which you can capture data as progress points along the way. The number of these data points and when you collect them will of course depend on the specifics of the project or experience and what you want to know in order to answer your guiding question(s). For instance, if it is a four week long project or experience, perhaps you are interested in tasks or activities that serve as data measures once a week or once every two weeks. Even a project or experience that is one day long can have data measures at different critical points throughout the time in which students are engaged. Regardless of how you design this aspect of your study, it will allow for more robust data collection and may reveal progress towards outcomes that support your findings in the end.  Not only do in-progress measures serve as ways to collect more data, they can also point to issues arising within the project or experience that may require intervention, allowing the researcher even more insight into learning processes and outcomes as the result. These tasks and activities can be carried out individually or in pairs or groups. Some examples of means to do this are:

  • Weekly reflections on how the project or experience is going, or what the students are experiencing,
  • Diary or journal entries in which students discuss the highs and lows, the challenges and breakthroughs,
  • Photos or videos,
  • Observations of students engaged in various project or experience activities,
  • Social media activity, for instance posts to Facebook or Twitter or blogs,
  • Summaries of project tasks or activities in which the students have engaged.

As your project or experience draws to a close, what are some final sources of data that you can collect that will aid in answering your guiding questions? Consider carefully what these could be and what they could tell you about the out-of-class learning that has taken place over the course of your project or experience. These final tasks or activities could be summative in nature or more reflective. Perhaps they require that your students return to their initial thoughts and ideas about the project or experience and how those have changed or been impacted as the result of their participation. Through this type of data measure, you provide students the opportunity to reflect on their own out-of-class learning, which has the potential to support, or maybe even refute, your own findings within the data. Some examples of final data measures include:

  • Final reflection writing activities,
  • A survey with Likert-scale items, open-ended prompts or both that revisit initial survey questions or ask completely new ones,
  • A focus group with your students,
  • Interviews with student participants,
  • A final project report or summary,
  • A presentation of the work that was carried out,

There is no doubt that SoTL provides a unique opportunity for researching and assessing out-of-class learning. Given that many university classes across the disciplines offer field experiences, service learning activities or special projects that take students out of the classroom in an effort to make real-world connections between content and practice, this kind of research can be used to provide insight into what students actually take away, both personally and professionally, from such experiences.  However, in order to do this, SoTL researchers must be purposeful in how they not only design the project or experience in which students engage, but also how they will capture data at the beginning, end and at various points throughout.


Leave a comment

Give students hints on the exam…but not for free!

Written by: Jerry Schnepp, Ph.D., Department of Visual Communication Technology at Bowling Green State University

I would like to share with you some details on  a project that allowed students to barter points for hints on an assessment. Here’s how point bartering works in an assessment context:

  • If a student does not know the answer to a question, she can click the “barter” button.
  • This allows her to trade a predetermined point value in exchange for a hint or part of the answer.
  • The student can use the hint to answer the question correctly, but for less credit than those students who did not use this process.

Thus, instead of answering incorrectly and receiving no points, the student can accept a point reduction and potentially find the correct answer. I developed a software program called Point Barter to facilitate these transactions (see image below for an exemplar question).

The Point Barter interface:

point barter

The concept of giving hints on exam questions is nothing new. I’m sure many of you do it in one way or another. That said, a computer program designed to facilitate the process is unique and has two potential advantages:

  • First, it’s completely automated once programmed.
  • Second (and perhaps more compelling), it provides an equitable distribution of hints. That is, strong students who do not need hints can answer correctly without bartering. They earn full points, while weaker students who could use help may choose to sacrifice some points in order to find the correct answer. Students who need a great deal of help may choose to barter multiple times on an assessment, in turn reducing their overall assessment score.

I developed Point Barter after I had been teaching as a graduate student for about ten years. I had just accepted a tenure-track position and I felt compelled to really examine my approach to teaching and assessment. I noticed that grade distributions tended to be bimodal (see image below), with students seeming either well-prepared or ill-prepared for their exams. Most of the time, however, we know that’s not exactly the case. There are plenty of students who study hard, only to find that they can’t remember what they had studied when asked to recall information. I wanted to find a way to give these students the nudge they needed to achieve success, but do so equitably.

A typical assessment distribution

dual modes.png

As I pondered how to do this, I thought about students requesting “hints” during exams in the past to trigger their memory. Upon reflection, I found the idea of rationing hints to be interesting as a learning support to help student recall and spark important memories.

And that’s where point bartering systems, such as the one I developed, seem to have impact. Students who have used the system have reported that often times they “know the answer, but need something to jog [their] memory.” A preliminary study indicated that students learn how to use the barter feature quickly. After using it for several exams, most prefer point bartering to traditional online testing systems such as Canvas, Moodle and Blackboard.

You’re probably thinking, “Of course students like it. It helps them to get better grades.” However, when we conducted a study comparing two sections of the same course, one that used Point Barter and one that did not, the median exam scores were similar. Because students who barter for hints must sacrifice points, scores tended to equalize across test takers.

These similarities in scores did not translate to final exams. Rather, the group that was able to barter points throughout the semester scored significantly higher than those who did not on the identical, cumulative final exam (see chart below). Since the course lectures, assignments and projects were identical for both groups, we assert that using point bartering as part of the assessment process during the semester promoted additional learning. If students were unsure about an answer, they could barter for a hint, thus obtaining new knowledge. Using point bartering, it is possible that students knew more about a given subject after the quiz than they did before, demonstrating potential learning during assessment.

Grade difference between class sections. The experimental group used Point Barter throughout the semester. The control group used Canvas (a non-point bartering assessment platform).

post test.png

After having read about the point bartering system I used, I’d welcome comments from the readers of the SoTL Advocate. Please consider commenting on the following questions, or asking different questions of your own:

  • Would you use a system like Point Barter?
  • Do you see any problems with it?
  • Do you think that point bartering is particularly suited for certain disciplines or types of assessments?
  • Do you see applications for point bartering outside of academia within a given discipline or field of practice?


Leave a comment

Call for Proposals: Banff Symposium on SoTL

Call for proposals: Banff Symposium on SoTL
Proposals due: May 8 2016
banff
The Institute for Scholarship of Teaching and Learning (SoTL) at Mount Royal University in Calgary, Canada invites you to submit proposals to our 7th annual Symposium on SoTL, Nov 10-12, 2016.  The Symposium is a practitioners’ conference dedicated to developing teaching and learning research, sharing initial findings, going public with results of completed projects, and building an extended scholarly community.  Proposals are encouraged from students, faculty, administrators, or community members committed to the systematic scholarly inquiry into aspects of teaching and learning in a higher education setting.
Conference theme: Learning in and Across Disciplines
Participants at previous Symposia have told us how much they value the connections they make across roles, disciplines, and institutions. We encourage presentations that demonstrate collaborations with students, with other instructors, and among multiple disciplines and contexts.
Conference tracks:
  • Research on teaching and learning – presentations on active or completed SoTL projects
  • Involving undergraduate students in SoTL – presentations on best practices or example projects where undergraduate students are acting as co-researchers
  • Teaching and learning with technology – presentations on the utility and impact of technology for teaching and learning
  • Collaborating beyond the single classroom – presentations on multi-class, interdisciplinary, or cross-institutional projects
  • Methodologies and innovative approaches to data gathering and analysis – presentations providing a ‘how to’ introduction to specific research methods and theoretical frameworks
  • Calls for collaboration, triangulation, and development (poster session only) – poster presentations that share early-stage research questions with the objective of establishing connections with like-minded researchers
Full conference details here: http://isotlsymposium.mtroyal.ca/
twitter: #ssotl16


1 Comment

SoTL as Women’s Work

Written by Kathleen McKinney, Cross Endowed Chair in SoTL, Illinois State University

This post consists of edited excerpts from the following article:

McKinney, Kathleen and Chick, Nancy L. (2010) “SoTL as Women’s Work: What Do Existing Data Tell Us?,” International Journal for the Scholarship of Teaching and Learning: Vol. 4: No. 2, Article 16.

In this essay on the field of SoTL, we reported on an exploratory, descriptive study of the levels of participation of men and women in various types of scholarship of teaching and learning (SoTL) activities. For the purposes of the study, we defined SoTL
as the evidence-informed study of teaching and learning by disciplinary specialists that is made public. Anecdotally, we had both noticed what appears to be disproportionate involvement of women in most SoTL activities. In addition, both of us have had other people express to us their curiosity about this apparent fact. Thus, in an exploratory and descriptive study, we looked at some existing data relevant to this issue. While we acknowledged that other factors (e.g., discipline, institutional context, and academic rank) may also affect participation in SoTL research and other activities, we focused on the gender of SoTL participants.

We considered various ideas in hypothesizing about our results: gender role socialization and structures and opportunities in disciplines and institutions (e.g., representation of women and men in various academic positions or institutions, discrimination, status and power). We expected the data to show a pattern of women being disproportionately involved in most SoTL opportunities relative to their actual representation among those who could participate in SoTL and other SoTL activities. More specifically, we believed that disproportionately larger percentages of women than men would be involved in self-selected SoTL activities, as well as in activities that are primarily self-selected but also involve some appointment or confirmation by others. We also believed the representation of men and women would be closer to proportional for the higher-status or higher-prestige SoTL opportunities that are primarily awarded or invited by others.

We began by finding existing data to help us estimate the representation of women and men with doctoral degrees and in various higher education academic faculty/staff positions in multiple nations as ‘baseline’ data. We then found and coded 25 other forms of existing data on the representation of women and men national and international SoTL activities. These activities included membership in a SoTL professional organization, holding leadership positions in SoTL organizations, presenting at a SoTL conference or event, serving on the editorial board for or publishing in a SoTL journal, and winning a SoTL award or being selected as a SoTL fellow or scholar.

Using that data, we found the following patterns:

  • Women are over-represented, relative to the numbers of men and women faculty/academic staff in higher education, in both ‘self-selected’ SoTL activities and in ‘primarily self-selected with other approval or confirmation’ activities.
  • The involvement of women and men was more representative to their numbers for activities in the ‘primarily invited, awarded, or selected by others’ SoTL category.

We noted the limitations to the research (it was a descriptive and exploratory look at the issue with some methodological weaknesses). Finally, we discussed some possible implications of these results for women and men, for the field of SoTL, and for the value and reward for SoTL. We wonder whether our findings would still hold today, many years after our existing data was found and coded. We welcome comments by blog readers on the full study and our ideas.