The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…


Leave a comment

Connecting Employer Demands with Student Perceptions

Written by: Michael Barrowclough and Michelle Kibler, Assistant Professors of Agribusiness in the Department of Agriculture at Illinois State University (mlkible@ilstu.edu)

We as researchers are challenged with facilitating learning of our discipline to students while preparing them to enter the workforce. The two are not mutually exclusive. We are interested in the demands employers place on new hires (often college graduates) within the agricultural industry and how students’ perceptions of those demands correlate. While gaining knowledge in a particular field of study is of high importance, what can be overlooked is the significance of the development of soft skills such as time management, dependability, and communication to name a few. Knowing the soft skills sought by employers is only half the battle as proficiency in those skills must often be learned through practice and repetition. At times, educators have struggled to bridge the gap between academia and industry (Neuman and Banghart, 2001; Thacker, 2002; Wright, 2002; Cox and King, 2006).

To help us bridge this gap, we developed a survey aimed at determining the skill(s) most preferred by employers, as well as the skill(s) employers find most lacking in new hires in the agricultural industry. The survey was distributed to professionals in the agricultural industry attending the 2017 Agricultural Career Fair at Illinois State University. A total of 71 surveys were completed, with 50 of the 52 companies attending the career fair participating. Survey questions focused on the respondent’s opinion regarding the importance of six skills in new hires:

  1. Applying knowledge/skills to the work environment,
  2. Being innovative/creative,
  3. Computer applications/Staying current on technology,
  4. Locating, analyzing, and evaluating information to solve complex problems,
  5. Oral/Written Communication, and
  6. Working with others in teams

These six skills were chosen following an extensive literature review in conjunction with results of previous research conducted by the authors (Barnett, 1997; Fallows and Steven, 2000; Lowden et al., 2011; Finch et al., 2013; Kibler and Barrowclough, 2016).

The survey used a choice-based method known as “Best-Worst Scaling” (Finn and Louviere, 1992; Louviere and Islam, 2008; Louviere et al., 2015). Participants were shown 10 different scenarios, with each scenario containing a list of three of the above mentioned skills. In each scenario, participants were asked to select which skill (of the three listed) they felt was “most important” and which skill (of the three listed) they felt was “least important”. This choice-based method has significant advantages over other survey formats (e.g., ratings scales) in that it allows for an individual’s strength of preference for multiple objects to be calculated over a defined measurement range (Louviere et al. 2013).

The skill employers found to be most important was Oral/Written Communication, followed by Applying knowledge/skills to the work environment, Working with others in teams, and Locating, analyzing, and evaluating information to solve complex problems. Being innovative/creative and Computer applications/Staying current on technology were found to be least important to employers attending the career fair.

With these results, instructors may choose to tailor existing course activities (assignments, group projects, presentations, etc.) or create new opportunities to enhance student abilities in these areas. By linking the skills that employers find “most important” to how developed employers find those skills in their new hires, we as educators may provide a classroom experience which better prepares Illinois State University agriculture students for employment in the highly competitive agricultural industry.

Using the information gained from the employer questionnaire, we plan to administer a similar questionnaire to undergraduate students in the Department of Agriculture, focusing on the following objectives:

(1) examine agricultural students’ perceptions concerning skills, knowledge, and abilities they believe employers find important in recent college graduates;

(2) elicit student reflections and comments on course activities that provided experience or practice regarding specific skills and abilities throughout the semester

We anticipate that by surveying both groups (employers and students) we may identify any potential gaps that exist between student perceptions and employer demands. Students may benefit by having a clearer understanding of the skills and abilities valued by employers. Through reflections on what course activities relate to or provide experience with various employer sought skills, students may be encouraged to consider the benefits these activities provide in addition to gaining subject knowledge.

Blog References:

Cox, S. and D. King (2006), ‘Skill Sets: An Approach to Embed Employability in Course Design’, Education and Training, 48(4): 262-274.

Finch, D., L. Hamilton, R. Baldwin, and M. Zehner (2013), ‘An Exploratory Study of Factors Affecting Undergraduate Employability’, Education and Training, 55(7): 681-704.

Finn, A. and J. Louviere, 1992, “Determining the appropriate response to evidence of public concern: The case of food safety”, Journal of Public Policy and Marketing, 11(1): 259-267.

Kibler, M. and M. Barrowclough (2016), ‘Career Fair Questionnaire’, Submitted to industry participants at the 2016 Agricultural Career Fair at Illinois State University.

Louviere, J., T. Flynn, and A. Marley, 2015, “Best-worst scaling: Theory, methods and applications”, Cambridge University Press.

Louviere, J. and T. Islam, 2008, “A comparison of importance weights and willingness-to-pay measures derived from choice-based conjoint, constant sum scales and best-worst scaling”, Journal of Business Research, 61(9): 903-911.

Louviere, J., I. Lings, T. islam, S. Gudergan, and T. Flynn, 2013, “An introduction to the application of (case 1) best-worst scaling in marketing research”, International Journal of Research in Marketing, 30(3): 292-303.

Neumann, B. and S. Banghart (2001), ‘Industry-University ‘Consulternships’: An Implementation Guide’, International Journal of Educational Management, 15(1): 7-11.

Thacker, R. (2002), ‘Revising the HR Curriculum: An Academic/Practitioner Partnership’, Education and Training, 44(1): 31-39.

Wright, J., L. Cushman, and A. Nicholson (2002), ‘Reconciling Industry and Academia: Perspectives on the Apparel Curriculum’, Education and Training, 44(3): 122-128.

Advertisements


Leave a comment

Pedagogical Content Knowledge, Language Awareness, and edTPA: ISU Language Teacher Candidates’ Feedback Practices

Written by: Susan Hildebrandt, Ph.D., Associate Professor of Languages, Literatures and Cultures at Illinois State University

feedbackWhat makes feedback effective for students and encourages their later learning? Few would argue that timely feedback is desirable, and that it should provide students with a path forward. In the language classroom, the language in which the feedback is expressed also matters. My study, supported by a 2017 SoTL Summer Mini Grant, focused on ISU Spanish student teachers’ linguistic choices as they provided feedback to their K-12 Spanish learners during student teaching. Using the world language edTPA assessment task, I compared high-scoring edTPA portfolios to low-scoring portfolios. With an eye to similarities and differences in demonstrations of language awareness, the study investigated whether participants used English and/or Spanish in their feedback, along with the quantity and characteristics of that feedback.

edTPA became consequential in Illinois in September 2015, and all teacher candidates in Illinois who wish to earn a teaching license must pass that high-stakes, standardized assessment. edTPA evaluates teacher candidates’ ability to plan, instruct, and assess K-12 student learning through an extensive portfolio submitted to Pearson for external scoring at a cost of $300. It is a graduation requirement in ISU teacher education programs and is used in a number of states to evaluate effectiveness of teacher education programs.

Pedagogical content knowledge (PCK, Shulman, 1987) develops over the course of a career, beginning during preservice coursework and continuing throughout later in-service teaching and learning experiences (Henze & Van Driel, 2015; Lortie, 1975). It is domain- and discipline-specific (Hashweh, 2013; Shulman, 1987), comprised of content knowledge, or the what of teaching, and pedagogical knowledge, or the how of teaching (Shulman, 1987). Language awareness, as defined by Thornbury (1997), is “the knowledge that teachers have of the underlying systems of the language that enables them to teach effectively” (p. x). Language awareness, a subset of teacher candidates’ PCK, proves critical in language teacher education courses.

Much time is spent in postsecondary world language pedagogy classes to teach future language teachers how to teach in the target language, or the language that is being taught (i.e., Spanish) (Hildebrandt & Swanson, 2016). That training prompts teacher candidates to use the target language in a way that promotes student uptake and develops their language proficiency. It also seeks to avoid teacher candidates reverting back to teaching the way that they were taught (Cruickshank, Metcalf, & Bainer Jenkins, 2009); that is, it seeks to help teacher candidates avoid teaching about languages instead teach in the language itself. Methods classes can help teacher candidates internalize constructivist teaching practices and apply them to their feedback practices (Sigler & Saam, 2006), but previously held dispositions can prevent that application of best practices (Cummins & Asempapa, 2013). The language chosen for student feedback is critical in second language teaching, and scaffolding teacher candidate feedback to students can provide a valuable aid in long-term changes in teaching practices (Hunzicker & Lukowiak, 2015).

Both qualitative and quantitative methodology were used in this study. The initial data set was composed of World Language edTPA scores from a ISU’s seven Spanish teacher education program completers during the 2015-2016 academic year. Analysis of participants’ assessment artifacts (e.g., rubrics, evaluation criteria, etc.) allowed easier comparison of the level of class, type of rubric used, rubric criteria, feedback type, language of the rubric and feedback, among other features. Using qualitative analysis, I also explored ways that the portfolios with high average assessment task subscores manifested language awareness, as compared to the portfolios with low average subscores. Three general categories, based on the portfolios, formed the initial structure for analysis: language awareness, knowledge of L2, and knowledge of learners. Results showed that most participants’ feedback was in English, rather than Spanish. In general, those participants who provided more feedback to students, no matter the language, scored better on the edTPA assessment task.

This project sought to explore ways of preparing ISU teacher candidates of Spanish to complete an edTPA portfolio to align with communicative language teaching practices, including using the target language nearly exclusively. Those practices are explored and put into practice during coursework, but do not seem to generalize to the edTPA portfolio constructed during the student teaching semester (Swanson & Hildebrandt, 2017), as was found in the present study. I will use the information gained to create more opportunities for ISU teacher educators to create effective, Spanish-language feedback to their K-12 students (Hunzicker & Lukowiak, 2015).

The full study can be found in a chapter called “Pedagogical Content Knowledge, Language Awareness, and edTPA” that will appear early next year an edited volume. Researching edTPA Promises and Problems: Perspectives from English to Speakers of Other Languages, English Language Arts, and World Language Teacher Education will be released in early 2018 by Information Age Publishing.

Blog References:

Cruickshank, D. R., Metcalf, K. K., & Bainer Jenkins, D. (2009). The act of teaching (5th ed.).New York, NY: McGraw-Hill Higher Education.

Cummins, L., & Asempapa, B. (2013). Fostering teacher candidate dispositions in teacher education programs. Journal of the Scholarship of Teaching and Learning, 13(3), 99-119.

Hashweh, M. (2013). Pedagogical content knowledge: Twenty-five years later. In C. J. Craig, P. C. Meijer, & J. Broeckmans (Eds.), From teacher thinking to teachers and teaching: The evolution of a research community (pp. 115-140). Bingley, UK: Emerald Group Publishing.

Henze, I., & Van Driel, J. H. (2015). Toward a more comprehensive way to capture PCK in its complexity. In A. Berry, P. Friedrichsen, & J. Loughran (Eds.), Re-examining pedagogical content knowledge in science education (pp. 120-134). New York: Routledge.

Hildebrandt, S. A., & Swanson, P. (2016). Understanding the world language edTPA: Research-based policy and practice. Charlotte, NC: Information Age Publishing.

Hunzicker, J., & Lukowiak, T. (2015). Engaging pre-service teachers –and their professor – in learning: A comparison of two literacy methods courses. Journal of Transformative Learning, 3(2), 52-83.

Lortie, D. (1975). Schoolteacher: A sociological study. Chicago: The University of Chicago Press.

Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1-22.

Sigler, E. A., & Saam, J. (2006). Teacher candidates’ conceptual understanding of conceptual learning: From theory to practice. Journal of Scholarship of Teaching and learning, 6(1), 118-126.

Swanson, P. & Hildebrandt, S. A. (2017). Communicative learning outcomes and world language edTPA:  Characteristics of high-scoring portfolios. Hispania, 100, 331-347.

Thornbury, S. (1997). About language. Cambridge: Cambridge University Press.

 

 

 


Leave a comment

Photo Documentation: SoTL Methods Series #4

Written by Jennifer Friberg, Cross Endowed Chair in SoTL and Associate Professor of Communication Sciences and Disorders at Illinois State University

Over the last five years, students from my academic department have traveled to six countries on two continents as part of our program’s study abroad experience. Offered for independent study credit, these study abroad experiences have transpired as “short term” opportunities (e.g., spring break or two weeks in the summer session) and have focused on cultural immersion, rather than disciplinary content knowledge. I led our most recent trip to Spain during spring break 2017 and was joined by two faculty colleagues and 32 students. Prior to this year’s trip, students self-reported gains in what Miller-Perrin and Thompson (2014) would consider internal learning (e.g., emotional, intellectual, and spiritual growth) as well as external learning (e.g., second language acquisition, intercultural learning, globalization, disciplinary knowledge). While this was lovely to hear, these impressions were anecdotal, as we had never systematically studied student learning that occurred as a result of engagement as a study abroad participant.

I decided to study this year’s students to measure the impact of short term study abroad for this cohort. Initially, I thought I would restrict my study design to traditional methods of reflection and content analysis. In the weeks prior to travel, I asked students to set three goals for themselves based on self-perceived areas of weakness or interest. Students were asked to keep track of their growth in these goal areas via reflective journals, which were analyzed carefully for evidence of progress or growth in their goal areas. Eighty-one of these goals fit into one of the following categories: taking chances, engagement, flexibility, gaining independence, archiving, budgeting, social interaction, or understanding culture.

Around the time that I was developing the plan for collecting data from travelers, I read about photo documentation, a method derived from visual sociology wherein researchers seek out patterns in photographic data collected using something called a shooting script. I was intrigued as to how I might introduce this new method into my study abroad project. I tried, and I was able to extend what I learned about my students in doing so. The remainder of this post is a description of my “rookie experience” with photo documentation. I am certainly not an expert, but I did enjoy exploring this visual methodology!

Photo documentation is a research method developed by Charles Suchar (1997). Specifically, “photo documentation is a method that assumes photographs are accurate records of what was in front of the camera when its shutter snapped – ‘a precise record of material reality’ — and takes photographs in a systematic way in order to provide data which the researcher then analyzes” (Rose, 2016, p. 310). The key to photo documentation is a shooting script which consists of “lists of sub-questions” (derived as topical to overarching research questions) which act as a guide for taking pictures connected to the topic of the research being conducted (Rose, 2016, p. 311). Photos are taken in accordance with a shooting script, then analyzed for categories and patterns via a systematic coding process.

In the case of the study I described above, I wanted to understand changes that occurred in students in self-identified areas of need/interest. Thus, in addition to reflective (written) journaling, I asked that students use the following shooting script to document learning visually before and during travel:

  1. What was I doing/seeing when I recognized that I had made progress toward meeting one of my goals?
  2. What things/people/experiences influenced this progress?

spain food 2In applying this shooting script, students were directed to take pictures to answer these questions as they were going about their study abroad experiences. After travel, they submitted three photos per goal to me, along with their written journals. I printed out all photos that students submitted and searched for patterns across goals and photos. I was able to find many! As an example, many students set goals to improve their understanding of different cultures. Looking at the photos submitted along with written reflection for this goal, I could see that students represented “increases in cultural understanding”  via photos of food (see right), religious symbols, architecture, or Spanish citizens they interacted with during their travels. Students who set goals to “take chances” maggierepresented growth in this area with photos of new foods they tried or photos of experiences where they conquered personal fears (e.g., communing with the macaques in Gibraltar after a prior bad experience — see left).

Overall, what I found interesting is the different stories these data told. While written reflections told me WHAT changes students had realized as a result of study abroad, the photographs told me HOW how these changes occurred. The combination of these different data types were powerful to tell the story of my students’ experiences in Spain and made a powerful case for the possibility of significant learning in a short amount of time.

I have always been intrigued with what the visual representation of learning might look like for different students. Using photo documentation helped me to see the possibilities of visually-based data in a way that I appreciate and hope to use again in the future!

Blog References:

Miller-Perrin, C. & Thompson, D. (2014). Outcomes of global education: External and internal change   associated with study abroad. New Directions for Student Services, 146, p. 77-89.

Rose, G. (2016). Visual methodologies: An introduction to researching with visual methods (4th ed.). Sage: Los Angeles.

Suchar, C. S. (1997). Grounding visual sociology research in shooting scripts. Qualitative Sociology, 20(1), 33-55.