A few weeks ago, the National Center for Education Statistics, the statistical arm of the U.S. Department of Education, released a report with the headline finding that there is no difference in the test scores of students attending charter and traditional public schools. The finding is technically correct but highly misleading.
NCES simply compared the average test scores of students who are enrolled in charter schools with those of children who attend traditional public schools. That analysis literally answers the question: Do the test scores of kids in charter and traditional public schools differ? The answer isn’t remotely interesting. The policy-relevant research question is: What is the difference in later outcomes for students who enroll in a charter school compared with the outcomes that the same students would have achieved had they instead attended a traditional public school? Answering that question requires research design.
Comparing average school scores alone simply cannot answer that policy-relevant question because charter and traditional public schools enroll fundamentally different students. As the NCES report also shows, on the one hand, students attending charter schools are more likely to be African American or Hispanic and be eligible for free or reduced lunch than children enrolled in traditional district schools, but on the other hand, they are less likely to be learning English or have a disability. More importantly, charter and traditional public school students also almost certainly differ in other ways that we don’t observe. For instance, students in charter schools have parents who were active enough to make an affirmative decision to enroll them in a school of choice. It isn’t obvious whether charter or traditional public schools serve more advantaged students. But the two sectors clearly have different populations.
Marcus Winters is a senior fellow at the Manhattan Institute, an associate professor at Boston University, and author of the new report, “Should Failing Schools Be Closed? What the Research Says.” Follow him on Twitter here.
Photo by PragasitLalao/iStock