To: Canadian Ministers of Education
From: Clare Halloran, Rebecca Jack, James C. Okun, and Emily Oster
Re: The Effect of Online Learning on Student Performance: Evidence from US States
Date: January 5, 2022
Students who spent a higher fraction of the COVID-19 pandemic receiving in-person instruction performed significantly better on standardized tests in the spring of 2021 than those who were taught mostly remotely, according to a study of scores in 12 US states with a wide range of approaches to moving to remote learning. With Canadian provinces now moving students to remote learning, they should look to the US to see the academic harms to students from remote learning.
In our NBER Working Paper Pandemic Schooling Mode and Student Test Scores: Evidence from US States we further report that the fraction of students achieving passing grades fell more in districts with greater shares of minority students and students receiving subsidized lunches (which is a common measure in the US of which schools are serving low-income families).
Pass rates among students in all types of learning environments were lower in the spring of 2021 than in previous years. The average decline in math scores was 14.2 percentage points. For students who were fully in person for the 2020–2021 school year, the decline was smaller, 4.1 percentage points. The average decline in English Language Arts was 6.3 percentage points, but it was 3.1 percentage points among those taught fully in person.
Our study found a wide disparity in the decline in pass rates among the dozen states surveyed: Colorado, Connecticut, Florida, Massachusetts, Minnesota, Nevada, Ohio, Rhode Island, Virginia, West Virginia, Wisconsin, and Wyoming. It ranged from a 32 percentage point drop in the math pass rate for Virginia students to a 2.3 percentage point drop in the English pass rate in Wyoming (see Figure for scores on math). These numbers may understate the decline in pass rates because test participation was down in 2021, particularly among students from groups that have historically scored lower, according to the researchers.
A more refined picture of the impact of remote learning emerges after considering the demographic makeup of school districts. Racial and ethnic composition correlates with pass rates on the English Language Arts exam, but not with those on the math test. Extrapolating from our findings, we estimate that a hypothetical district with no Black or Hispanic students would experience a 4.3 percentage point drop in the pass rate for English Language Arts if it moved from fully in-person to exclusively hybrid or virtual learning. For a district in which half the students are Black or Hispanic, the analogous pass rate is projected to fall 9 percentage points.
We also relate declines in pass rates for the English test and, to a lesser extent, the math test to the economic well-being of students in each district. For a hypothetical district with no students receiving subsidized or free lunches (that is, a high-income area), we predict that moving from fully in-person learning to fully hybrid or virtual instruction would reduce the pass rate for math and English tests combined by 2.9 percentage points. In a district in which all students are eligible for subsidized or free lunches (a low-income area), we predict that the same instructional change would reduce the pass rate by 10.3 percentage points.
Large urban school districts with a higher share of Black and Hispanic students were more likely to rely on hybrid or virtual instruction. These districts also averaged lower pre-pandemic baseline test scores. Test scores are but one measure of academic performance. But the evidence is clear on at least this measure that virtual or distanced schooling modes cannot support student learning in the same way as in-person schooling. Canadian governments should have this evidence in mind as they consider over January whether to resume in-person schooling.
Clare Halloran is Associate Director of the COVID-19 School Data Hub at Brown University, Rebecca Jack is a PhD Student at the University of Nebraska-Lincoln, James C. Okun is a Pre-doctoral Research Fellow at MIT Economics, and Emily Oster is a Professor of Economics at Brown University.
To send a comment or leave feedback, email us at blog@cdhowe.org.
The views expressed here are those of the authors. The C.D. Howe Institute does not take corporate positions on policy matters.
