56 - Exactly How (In)Accurate Are Undergraduate Student Self-Reports?

Poster Board Number: 56
There are separate poster presentation times for odd and even posters.
Odd poster #s – first hour
Even poster #s – second hour

Abstract Body : Introduction and Objective
Educational research commonly uses student self-reports to collect data that will inform future teaching choices. However, we know from previous research that student self-reports are often inaccurate. Having inaccurate student self-reports has the potential to mislead educators as they attempt to make decisions to improve their classroom. The purpose of this study was to determine exactly how inaccurate student self-reports are, and if administering a metacognitive survey has any influence on student study habits and outcomes (i.e., grades).

Materials and Methods
Anatomy A215 is a large undergraduate course at Indiana University. During Fall 2021, the course adopted a flipped-classroom format in which students viewed recorded lectures online before attending in-class active learning sessions.
Following students’ second lecture exam (out of four), students completed a self-report reflection including questions on how often they attended active learning sessions and watched lecture. Student responses were gathered and then compared with analytics data of their active learning attendance as well as recorded lecture views. Statistical analyses were run using SPSS.

Results
Survey responses and attendance analytics demonstrated a strong correlation (r=0.720, p< 0.001) with students who reported attending active learning sessions having significantly greater attendance than students who reported occasionally or not attending (p< 0.001). However, there was not a statistically significant difference in attendance between the students who reported either occasionally or not attending active learning sessions (p=0.228). Of all the students who reported always/almost always attending active learning sessions (n=223), 23% of them actually attended less than 60% of active learning sessions. Furthermore, dependent samples t-tests revealed a statistically significant drop in both attendance and exam scores after the self-report survey was administered (p< 0.001 for all response groups).
When considering the videos, a moderate correlation was found between survey responses and total lecture watch time (r=0.449, p< 0.001). In addition, there was a statistically significant difference in watch time for those who reported walking lecture once (average watch time 63.4%) and those who reported not watching lecture (average watch time 30.3%; p< 0.001). Both response groups had a statistically significant decrease in average watch time after receiving the self-report survey (p< 0.001 for both response groups).

Conclusion
While many students were accurate with their responses, the sizable number of inaccurate reports is enough to call into question the usefulness of student self-reports. For example, reporting occasionally attending vs. not attending active learning sessions had no bearing on actual attendance. In addition, a sizable component of students who reported attending active learning sessions only attended < 60% of sessions. The administration of a self-report survey did not improve student outcomes, but other factors may have resulted in decreased grades and attendance as the semester progressed.

Significance/Implication
Student self-reports, even if they are just reporting on attendance and lecture, have enough inaccuracy to warrant extreme caution when using them to inform teaching practices.