Analysis of the prediction ability of a University self-evaluation test: statistical learning methods for predicting student performance

Abstract

The School of Economics and Management of the University of Florence uses a self-evaluation test as an instrument to assess the competencies of candidates who want to enrol in the three-year degree program. The aim of this study is to assess if the selfevaluation test scores give a gain in predicting the student performance when added to available student characteristics, such as the high school career. The student performance is measured by three binary indicators based on the number of credits gained after one year. For each binary outcome, the prediction is carried out using both logistic regression and random forest, using two alternative sets of predictors: (i) student characteristics; (ii) student characteristics and test scores. The predictive ability is assessed using 10-fold cross-validation. The main finding of the analysis, which refers to the academic year 2014/2015, is that the self-evaluation test scores do not help in predicting student performance once student characteristics are properly exploited.