Introduction

Where were you when everything shut down because of the COVID-19 pandemic? When the restaurants closed their doors, when the soccer practices were canceled, when the streets became silent?

For the youth of our country, the spring of 2020 was the abrupt start of years of online learning, affecting more than 70 million students nationwide (Klosky et al.). More than halfway through the school year, parents, teachers and students alike were forced to adapt to a new virtual environment. Families also grappled with their children spending more time at home and less time learning in school. The pandemic undeniably impacted youth across the country, but how big was the impact of COVID-19 on academic performance? Did pre-pandemic educational gaps worsen, and did our lowest-performing students fall even further behind? Were some school districts unable to adopt the alternative instructional models due to a lack of resources? Our project aims to answer these questions by studying the change in academic performance before, during and after the pandemic, focusing on the relationship between students’ test scores and socioeconomic factors that may have impacted them.

Using data from the Stanford Education Data Archive (SEDA) and the U.S. Department of Education’s Education Stabilization Fund (ESF), we aim to analyze how the pandemic affected standardized test scores of third to eighth graders between 2019 and 2022. We consider factors such as geography, race, socioeconomic status, state education policy, and funding on reading language arts (RLA) and math score changes. The SEDA dataset was created by researchers at Stanford to provide insight into average academic achievement per school district before and after the pandemic. It pulls state standardized testing scores and proficiency benchmarks, then uses statistical models to link state data to National Assessment of Educational Progress (NAEP) test data. This standardization process is necessary to force comparison since each state uses different tests and proficiency thresholds. However, this makes the data less intuitively interpretable: the resulting estimates are scaled such that 0 is equal to the average of the national NAEP average and 1 unit is equal to one NAEP point. We explain the dataset as well our own cleaning steps in more detail on our Data page.

Created by the Coronavirus Aid, Relief, and Economic Security (CARES) Act in March 2020, the Education Stabilization Fund invested more than $276 billion into state COVID-19 recovery efforts specifically for education. The Department of Education publishes how ESF funds are allocated by state, so we analyzed this dataset in conjunction with the SEDA dataset in order to examine the impacts of federal funding on educational success after the beginning of the pandemic.

Place in Literature

Much research has been conducted in regards to the effects of COVID-19 on education systems, with people looking into its impact through various angles. To guide our research and gain a better understanding of the impact of COVID-19 on education, it is critical we consider these past findings.

COVID-19 brought about an increased rate of remote learning, which some categorize as bringing a new form of flexibility and accommodation unbeknownst to traditional, in-person learning. Notably, however, there is still a large consensus on the negative impacts of COVID-19 on education, with reasons ranging from the difficulty of adapting to technology to increased student worry and a lack of student engagement. For instance, many have found that COVID-19 exacerbates disparities, such as a lack of reliable Wi-Fi and digital literacy among certain groups – factors that would be necessary for student success in a period of remote learning (Klosky et al. 2022). Moreover, literature has shown that COVID-19 brought about an increase in academic worry among students. According to a 2022 article about COVID-19’s impact on Georgian high school students, 66% of sampled students were worried about their academic career as a result of the pandemic and the turn to virtual learning (Ashta et al. 2022). Overall, research has shown that COVID-19 and remote learning has harmed educational outcomes. However, how much exactly COVID-19 has affected education, as well as what factors further exacerbate these effects, are still left unanswered at a state by state level. We aim to fill in these gaps.

Past studies have given light to this. In their 2023 article on test score recovery following COVID-19, Clare Halloran and their colleagues found significant declines in test scores by the end of the 2020-2021 academic year – but 20% of english language arts scores and 37% of math scores were recovered by 2022 (Halloran et al. 2023). Furthermore, the authors relate these drops and recoveries to state-level policies such as the federal American Rescue Plan Elementary and Secondary School Emergency Relief Fund – but they are unable to clearly pinpoint specific factors that led to learning losses and recovery.

Now, we aim to expand upon these efforts by looking at other state policies and the acknowledgement of disparities within socioeconomically disadvantaged groups in order to paint a better picture of how and why COVID-19 negatively impacted the education system and national test scores. We take a special consideration when it comes to socioeconomically disadvantaged groups as we recognize that these communities have faced educational disadvantages historically. Past literature has looked at the role of COVID-19 on students of color, with a 2022 article finding that Hispanic students in rural Georgia felt less “confident” and more concerned about their academic career than their white counterparts (Ashta et al. 2022). The study also found that students eligible for reduced lunch, non-honors, and low-attendance students were more worried about their academic success than their counterparts (Ashta et. al 2022). Such disadvantages go beyond COVID-19, as a 2018 article found a correlation between poverty and lower academic achievement (Morrissey & Vinopal 2018). Thus, we find it necessary to expand upon test score changes in relation to race, ethnicity, and class. 

There is still much that is left unsaid within the current literature. In order to guide future methods of instruction as well as policy, it is crucial we expand upon the current literature.

Significance

Our project will provide insight from a retrospective point by examining the impact of COVID-19 on education within the past four years. By analyzing several factors that may go into academic performance, we want to discover what is significant for youth learning outcomes in order to give improved guidance should something like this happen again.

Additionally, we hope that our project can shed light on any increased educational inequalities stemming from the pandemic. This is not only crucial to learning what we can do next time, but it is also imperative to understanding how we can take action now to help those who have fallen behind and have not fully recovered.

Our insights will help us identify what kind of government aid would be most beneficial to students and also which communities would benefit the most from aid. If there is a next time, we hope to use this project to evaluate how we can improve emergency measures to ensure student success across varying socioeconomic backgrounds.