The SoTL Advocate

Supporting efforts to make public the reflection and study of teaching and learning at Illinois State University and beyond…

Study Design and Data Analysis in SoTL

1 Comment

Written and compiled by Jennifer Friberg, Cross Endowed Chair in SoTL and Professor of Communication Sciences & Disorders at Illinois State University

In June, I visited the University of South Alabama (USA) and worked with Raj Chaudhury and Sue Mattson to get a group of faculty started with their year-long SoTL Academy efforts. Approximately 30 faculty from across USA’s campus came together to learn about SoTL and plan a SoTL project. We spent two days together in workshops and consultations and all participants left with a draft plan for SoTL work they hoped to conduct this current academic year.

This was the second year I was able to join the USA crew for this two-day educational and research development event. Sue and I agreed that a resource that would be valuable for the USA faculty for the second iteration of the SoTL Academy would be something that helped social science-oriented researchers see how SoTL might dovetail with concepts and ideas they already understood well. Thus, the following grids focused on descriptive, correlational, and experimental/quasi-experimental design were drafted and used in discussions about how SoTL might look like participants’ own disciplinary research — and how it might not. This resource is being shared here now, in the hopes that others might find this information valuable, as well.

Should you wish to obtain a copy of this information in PDF form, please feel free to email me at jfribe@ilstu.edu. I’m happy to share!

Descriptive Research
Description of Study Design Descriptive research characterizes a group of people, a context, or a phenomenon. These studies do not seek to establish a causal relationship; rather, they provide information about “what is” occurring or being observed regarding the focus of study.

Descriptive studies include observational, case study, and survey methods.

Exemplar SoTL Projects
  • Survey students’ re: practices in using print vs. online textbooks to support learning.
  • Observe how students’ use of technology in the classroom impacts attention span.
  • Study high achieving students in a course to predict practices/variables of success to share with future students.
Qualitative Analysis Options Qualitative data in a descriptive study is reported as narrative, reflection, open-ended response, field note, etc. Such data will need to be further analyzed for themes, categories, or patterns.

Common qualitative approaches in descriptive SoTL research include: case studies, action research processes, analytic induction, ethnography, comparative analysis, frame analysis, grounded theory, and interpretive phenomenology, among others.

Quantitative Analysis Options Quantitative data in a descriptive study is often reported in the form of descriptive statistics (e.g., mean, median, mode) along with standard deviations. Statistics might be used here, depending on the data collected and the topic being studied.

These data might emerge from test scores, grades on a course assignment or project, survey data, or frequency data.

 

Correlational Research
Description of Study Design Correlational research seeks to determine whether a relationship exists between two or more variables, but cannot determine if one variable causes another. Variables aren’t manipulated; rather, they are observed to determine any relationship that might exist between them.

Note that some sources identify correlational research as a quantitative-only subset of descriptive research, as some descriptive research might suggest a correlation found via grounded theory or other qualitative methods of research.

Exemplar SoTL Projects
  • Determine the relationship between number of hours studying and success on a quiz/test.
  • Identify whether there is a link between the use of peer editing and performance on a writing assignment.
  • Understand whether the use of social media helps students to summarize course content effectively.
Qualitative Analysis Options Qualitative data analysis is not undertaken for correlational research, as numerical data is needed to calculate a correlation coefficient.
Quantitative Analysis Options Correlational research is a quantitative method of inquiry. Correlation can only be determined for quantifiable data. These are data in which numbers are meaningful, usually quantities of some sort. It cannot be used for purely categorical data, such as gender, brands purchased, or favorite color.

Statistics are used to determine a correlation coefficient to identify positive, negative, or zero correlation. One thing to keep in mind is that any identified correlation does not mean that one variable caused the other to react. Instead, correlations simply define that a relationship exists.

 

Experimental/Quasi-Experimental Research
Description of Study Design Experimental and quasi-experimental research designs seek to manipulate one variable and control all others to investigate cause/effect relationships. All participants are assigned to either a control or experimental group. An intervention is applied to the experimental group. The control group has no intervention applied.

The key difference between experimental and quasi-experimental designs is the concept of randomization. If participants are assigned to control and experimental groups randomly, the research design is experimental. Non-random group assignment yields a quasi-experimental research design. True experimental research is considered the gold standard of research by many researchers, because random group assignment leads to optimal internal validity. In situations where random group assignment is not possible or ethical, quasi-experimental designs offer an alternative that allows the research to continue and still produce valid results.

Almost no SoTL qualifies as truly experimental in nature due to inherent ethical and logistical characteristics of SoTL that makes this type of research difficult to conduct (e.g., true randomization). One of the most common quasi-experimental designs for SoTL research is the pre-test/post-test with no control group design.

Exemplar SoTL Projects
  • Does the use of simulated patients help nursing students improve observational skills?
  • Do architecture students who initially design structures by hand understand the concept of space more deeply?
  • Do history students exposed to guided reading demonstrate a deeper understanding of historical imagination?
Qualitative Analysis Options Experimental and quasi-experimental designs may yield data that is descriptive (e.g., surveys, interviews, observations) that require qualitative analyses. Similar to information provided above for descriptive research, any qualitative data will need to be further analyzed for themes, categories, or patterns.

Common qualitative approaches to data analysis in SoTL include: case study, action research processes, analytic induction, ethnography, comparative analysis, frame analysis, grounded theory, and interpretive phenomenology, among others.

Quantitative Analysis Options Experimental design lends itself to more straightforward and simpler types of statistical analysis. Primarily due to the lack of randomization, quasi-experimental studies usually require more advanced statistical procedures. Quasi-experimental designs may also utilize surveys, interviews, and observations which may further complicate the data analysis.

Quantitative analysis requires several steps. First numeric data is assigned a level (nominal, ordinal, interval, or ratio). Next, descriptive statistics are calculated for data (e.g., means, standard deviations). For some studies, descriptive statistics may be adequate; however, if you want to make inferences or predictions about your population, inferential statistics (e.g., t-test, ANOVA, regression) may be indicated.

Blog References:

Bishop-Clark, C. & Dietz-Uhler, B. (2012). Engaging in the scholarship of teaching and learning: A guide to the process and how to develop a project from start to finish.        Stylus: Sterling, VA.

Campbell, D. T. & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Cengage: Boston.

Cresswell, J. W. (2012). Qualitative inquiry and research design: Choosing among five approaches (3rd ed.). Sage: Thousand Oaks, CA.

Gurung, R. A. R. & Wilson, J. H. (Eds.). (2014). Doing the scholarship of teaching and learning: Measuring systematic changes to teaching and improvements in learning. Jossey-Bass: San Francisco.

 

 

Advertisements

One thought on “Study Design and Data Analysis in SoTL

  1. Pingback: Sources, Types, and Analysis of Data in SoTL | The SoTL Advocate

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s