Speaker: Rebeckah Fussell, Cornell University
Abstract: Advancements in natural language processing, including large language models (LLMs) have enabled education researchers to analyze open-response data efficiently and at scale. In my research, I aim to transform education research by leveraging LLMs to expand analysis of rich data sources (e.g. student writing) while ensuring that the methods employed at all stages of analysis are high-quality, trustworthy, and fair. I will present results from a recent study that compares LLM-based and traditional machine learning methods for measuring the prevalence of experimental skills in students’ written lab notes over time. In this study we fine-tune two different LLMs and compare the performance of these two fine-tuned models to a traditional machine learning approach (logistic regression) as well as a prompt-only approach (i.e. asking an LLM to identify experimental skills in a sentence without providing context or training). I will discuss implications from this comparison study and the broader outlook for LLMs in education research.
Hosted by: Physics and Astronomy
Additional Information can be found at: https://heellife.unc.edu/event/11871614