A team of economists from elite universities recently published a study suggesting that parents can’t truly assess school quality. Profiling the paper, The Atlantic notes that “if these results hold up under scrutiny by other economists … they could prove problematic for school-choice advocates.” But fortunately for them, this study merely shows that professors are far more ignorant than parents.
“Parents care about more than test scores.”
The study, published by the National Bureau of Economic Research, examines the preferences revealed by families’ choices when applying to New York City high schools. The authors found that families can identify and select quality schools that drive better outcomes for their children. So how is this happy story evidence of failure?
When the economists attempted to control for peer quality, they found that parents couldn’t necessarily spot the difference between schools that are more or less effective at boosting test scores, graduation and college enrollment. One rationale for school choice is that competition provides an incentive for school improvement. Presuming that they have managed to properly disentangle peer quality and school effectiveness, the authors suggest that their results imply that “school choice programs may generate stronger incentives for screening than for improved school productivity.”
The paper will soon be peer-reviewed by Ph.Ds who will probably pronounce it technically proficient. But even a pre-teen could spot the gaping logical hole, and perhaps illustrate it by way of a parable:
A scientist sets out to study whether little girls know how to pick flowers. After taking copious notes, he shakes his head and tells them, “Well, you girls sure know how to pick pretty flowers that make beautiful bouquets, but I’m sad to say you don’t know how to pick flowers at all.” Downcast, a girl asks, “How can that be?” The scientist explains, “You could have picked flowers of similar color and shape that would have lasted longer in a vase.” Abashed then perplexed, another girl asks, “Can you smell, Mr. Scientist?” “Of course,” he replies. “Well, this bouquet smells delightful,” she states. “Did you think about that?” “No,” he admits. She responds, “Wait, you designed a fancy flower experiment that ignored the sense of smell, and you’re telling us that we don’t know what we’re doing!?”
In a similar way, this study was almost literally sense-less. Parents care about more than test scores. They care about whether their children feel safe or not; whether they feel challenged and engaged or not; whether teachers love to teach them or not. The economists had the choice to study these factors or not. And they chose not.
New York City has administered a school climate survey to students, teachers and parents for the last decade, asking them about safety, engagement, morale and much more. The data were there, and the data were ignored. Co-author Parag Pathak of MIT told The Atlantic that while he understands why his finding “might be uncomfortable for school choice advocates … we have to go where the data leads us.” But his team reached its finding after ignoring the data, even though the paper admits that they “cannot rule out the possibility that preferences are determined by effects on unmeasured outcomes. Parents may be sensitive to school safety or other non-academic amenities.” They could not rule it that possibility, nor it seems could they be bothered to explore it.
Exploring it could very well have exploded their thesis. If parents can perceive and do value school safety, student engagement, teacher morale and community involvement, then school choice would give schools an incentive to improve all of those things. And according to a study published earlier this year by Temple University’s Sarah Cordes, that’s exactly what’s happening as New York City’s charter schools force traditional public schools to compete.
Cordes studied the traditional factors and found that charter competition, especially from nearby charter schools, raised test scores, increased attendance and decreased students being held back in nearby traditional public schools. But her study also broke new ground by incorporating data from the NYC School Survey. She found improvements in school safety, academic expectations, mutual respect and general school cleanliness and order.
It seems quite probable that those improvements occurred because school leaders realized that in order to retain and attract students, they’d have to offer a better school experience. If it’s true that parents can spot and do value those things, that would be powerful evidence that when it comes to school choice the “market” works.
On the other hand, if parents can’t really tell the difference between a safe school and one that’s dangerous, between a school where students are engaged and one where they’re bored, that would indeed be a profound problem for school choice.
But contrary to the credulous press coverage, this study poses no such problem. It ignores essential data, then alleges parental ignorance. And it reflects far more poorly on these elite economists than on New York City parents.
This piece originally appeared at RealClearEducation
______________________
Max Eden is a senior fellow at the Manhattan Institute. Follow him on Twitter here.
This piece originally appeared in RealClearEducation