Dissertation Defense

Title: Exploring Automated Essay Scoring Models for Multiple Corpora and Topical Component Extraction from Student Essays

Date: 2020-12-02

Host: School of Computing and Information, University of Pittsburgh

Speaker: Haoran Zhang

Summary and thoughts:

For this week’s talk, I attended the dissertation defense of a PhD student from the Department of Computer Science in the School of Computing and Information. Haoran Zhang defended his dissertation related to automated essay scoring using multiple datasets. Automated Essay Scoring (AES) aims to reduce the effort of manual essay scoring by automating the process with an objective approach. The speaker presented 3 models designed to grade essays based on topical components in the text. Using 2 datasets, one for holistic scoring and one for dimension scoring, the speaker implemented and compared three different models for essay scoring - feature based model, neural model and a hybrid model. Due to the characteristics of the datasets used, all models were designed to score source-based essays i.e. essays where a source text is involved and evidence from the source must be used to support the essay.

Model 1 aimed to improve the performance of an existing feature-based machine learning model using word embeddings for the dimension scoring task. Model 2 used co-attention based neural networks for source-dependent essay scoring. It overcame the limitation of model 1 where the need for human input was eliminated. The third model combined the neural network with hand-crafted features to achieve better performance in the task with both datasets. The results showed that the feature-based model outperforms its baseline, but a stand-alone neural network model significantly outperforms the feature-based model. Additionally, a hybrid model integrating the neural network model and hand-crafted features outperforms its baselines, especially in a cross-prompt experimental setting. It was an interesting talk as well as good experience to attend a dissertation defense to get an idea of the scope of work required for a PhD defense.