Manhattan Institute for Policy Research.
search  
 
Subscribe   Subscribe   MI on Facebook Find us on Twitter Find us on Instagram      
 
   
 

Civic Report
No. 12 August 2000


The Effect of School Choice:an Evaluation of the Charlotte Children’s Scholarship Fund

Jay P. Greene
Senior Fellow, The Manhattan Institute for Policy Research

Executive Summary

Does providing low-income families vouchers or scholarships with which they can select a private school improve student achievement? The evidence from the Children’s Scholarship Fund (CSF) program in Charlotte suggests that providing low-income families with scholarships has significant benefits for those families. This finding is consistent with the results from similar evaluations of scholarship programs in New York, Washington, D.C., and Dayton, Ohio as well as the results of evaluations of publicly funded school choice programs in Milwaukee and Cleveland.

The main findings from this evaluation of the Charlotte CSF Program are:

  • Receiving a scholarship to attend private school improves scores on standardized math tests by between 5.9 and 6.2 national percentile ranking points, depending on the type of analysis performed.
  • Receiving a scholarship to attend a private school improves scores on standardized reading tests by between 5.4 and 7.7 national percentile ranking points, depending on the type of analysis performed.
  • Parents were asked to assign their child’s school a letter grade, A through F. Nearly twice as many choice parents gave their child’s school an A (53%), compared to the public school parents (26%). Choice parents were also nearly twice as likely to report being “very satisfied” with virtually all aspects of their children’s school: location, safety, teaching quality, course content, class size, facilities, student respect for teachers, information on student progress, religious observance, parental support for school, discipline, clarity of school goals, teamwork among staff, teaching moral values, academic quality, and teacher respect for students.
  • Roughly two in five students would give their choice school an A compared to 32% of public school students. When students were asked how they feel about going to school each day, 24% of the public school students said that they did not want to go compared to 9% of choice students. And 24% of non-scholarship students agreed that they did not feel safe at school compared to 9% of choice students.
  • Parental reports confirm student perceptions about safety at school. More than a third of public school parents reported problems with fighting in school (36%) compared to 16% of choice parents. One-quarter of public school parents reported problems with racial conflict compared to 12% of choice parents. 22% of public school parents reported problems with guns or weapons at their children’s elementary schools compared to 11% of choice parents. And 25% of public school parents reported problems with destruction of property at school compared to 12% of choice parents.
  • Because the private schools examined operate with far less money per pupil than do the public schools, it is not surprising to discover that the private schools have more sparse facilities and fewer services to offer. For example, only 70% of choice parents described their school as having a library compared to 90% of the public school parents. Only 63% of choice parents said that their school had a gym compared to 91% of public school parents. Only 71% of choice parents said that their school had a cafeteria compared to 89% of public school parents. Parents also reported fewer school services at the private schools. Only 18% of choice parents said that their school had a program for students learning to speak English compared to 50% of public school parents. Only 49% of choice parents said that their school had a program for learning disabilities compared to 71% of public school parents. Only 51% of choice parents reported programs for gifted students at their schools compared to 72% of public school parents. Choice parents were also less likely to report that their school had a counselor, nurse, music program, art program, or prepared lunches.

The Charlotte CSF Program successfully targeted disadvantaged families. In general, choice schools were accepting students with scholarships who were considerably more disadvantaged than typical students in Charlotte. Three-quarters of the choice students were African-American, while a little more than one-third of all students in the Charlotte-Mecklenburg school district are African-American. As of 1990 the average family income in Charlotte was nearly $34,000, almost $10,000 more than the average family income of choice students 10 years later. Almost one-third (32%) of choice families report that they receive some kind of public assistance, such as food stamps or welfare, while the 1990 census reports that only 5% of households in Charlotte were on public assistance. And even after one year of the scholarship, choice students were still scoring well below the national average on standardized tests (although they were scoring significantly better than they would have had they not received the scholarship). There is no evidence to support the claim that the private schools were “creaming” the best students or “dumping” those students whom they found undesirable.

The private schools accepting scholarship students were smaller and had smaller class sizes, on average, than the public schools. But small class size does not “explain” the higher student achievement observed in private schools. Adding class size to the multivariate model predicting student test scores shows that class size has no effect on student achievement in our sample.

 

The Effect of School Choice: An Evaluation of the Charlotte
Children’s Scholarship Fund Program

Introduction

Does providing low-income families vouchers or scholarships with which they can select a private school improve student achievement?1  The evidence from the Children’s Scholarship Fund (CSF) program in Charlotte suggests that providing low-income families with scholarships has significant benefits for those families. This finding is consistent with the results from similar evaluations of scholarship programs in New York, Washington, D.C., and Dayton, Ohio as well as the results of evaluations of publicly funded school choice programs in Milwaukee and Cleveland. The findings of those studies have been summarized and discussed elsewhere.2  This report will focus on presenting the results from Charlotte.

Research Design

The CSF program offered partial scholarships to low-income families in Charlotte with a maximum value of $1,700 to attend private schools in the 1999-2000 academic year. To ration limited funds, scholarships were awarded by lottery to families that had completed an application process. This study examined only students enrolled in grades 2 through 8. In that age group, 388 students had been awarded scholarships by lottery and were enrolled in private school, 342 students were not offered scholarships by lottery, and 413 students had won the lottery to receive a scholarship but did not enroll in private school. All of these students and their parents were sent invitations to attend four testing sessions on a Saturday or Sunday between March 18 and April 30, 2000, where parents completed surveys while students took the Iowa Test of Basic Skills survey version. Older students also completed a survey.

Families whose children were not using scholarships were offered $20 and an opportunity to win a new scholarship as incentives to participate and to defray the transportation and other expenses involved. Families whose children were using scholarships were simply asked to participate without compensation. Despite these relatively modest incentives, our response rate was quite good. Of the 1,143 students who were sent invitations to attend a testing session, 452, or 40%, participated in the study. The participation rate among the students who won the lottery and were using scholarships, whom we will call “choice students,” was 53%. The participation rate among the students who applied but failed to win a scholarship in the lottery, whom we will call “control students,” was 49%. The participation rate among the students who won a scholarship but did not use it to attend a private school, whom we will call “non-complying students,” was 20%.

Various explanations account for the level of participation. The contact information available for all students was over a year old. Given the high mobility of urban, low-income populations, it is likely that many invitations never reached their target. In addition, we only offered four testing opportunities on Saturdays or Sundays, which may not have accommodated the work and social schedules of a number of families. Other factors that may have influenced participation include transportation issues, family motivation, and student cooperation with sacrificing a weekend day to take a standardized test.

These obstacles to participation were obviously most severe among the group that we call non-complying students. Many of those families did not use the scholarship that was offered to them because they moved, exacerbating the difficulty of inviting them to participate in this study. Other students who were offered scholarships but did not use them (and thus did not “comply” with a lottery research design), may have declined the scholarships because they obtained access to a desired public school, such as a magnet school or other public school choice program. If these students were doing well in their public school they would have little reason to participate in the study where the primary incentive was the opportunity to win a private school scholarship. Other students may not have used a scholarship that was offered because they were unable to find a satisfactory private school. Yet other students did not use a scholarship that was offered because their families did not have the financial resources to pay the tuition charges above the $1,700 value of the scholarship. Families that do not believe that they will be able to use a new scholarship are unlikely to be enticed by an offer of a scholarship to participate in the study.

Non-compliance and non-participation are issues in all evaluations, including random-assignment or lottery based studies, such as this evaluation and most medical studies. People are always free to cease cooperating with researchers and they are always free to refuse the treatment they are offered. Lotteries in research do not ensure identical treatment and control groups, but they certainly help get closer to achieving comparable groups than other methods of selecting subjects. To the extent that non-compliance and non-participation produce non-identical treatment and control groups, the differences can be adjusted statistically with little difficulty, as was done in this study.

Comparability of Groups

All applicants for scholarships were asked to provide their family income at the time of application. More complete demographic information was collected during the testing sessions, but, as noted, not all applicants participated in the study. By looking at the income information provided at the time of application we can see a number of things: 1) the lottery produced two groups that were not significantly different in income (this helps confirm that the lottery was properly conducted); 2) those applicants who participated in the study had somewhat higher incomes than those that did not; and 3) the differences between the incomes of study participants and non-participants are roughly equal for lottery winners and lottery losers as well as for choice, control, and non-complying students. In other words, while those who participated in our study differed somewhat from those who did not, those differences do not appear to have biased the comparability of our groups.

The family income of applicants who won the lottery to be offered a scholarship was $23,449 compared to $23,689 for those who lost the lottery. The difference in income is not statistically significant, helping to confirm that the lottery was fairly conducted. The family income of students who participated in the study was $25,313, which is significantly different from the $22,441 reported at the time of application for those families who did not later participate in testing. This gap of roughly $3,000 between participants and non-participants exists among those who won the lottery (combining the choice and non-complying students) as well as among those who were in the control group. Lottery winners who participated in the study had average family incomes of $25,323, while lottery winners who did not participate had average family incomes of $22,517. Control group students who participated in the study had average family incomes of $25,297, while control group students who did not participate had average family incomes of $22,215. Whatever factors influenced participation in the study appear to have operated equally on lottery winners and lottery losers.

This claim is further supported by the demographic similarity of the treatment and control groups who participated in the study and completed our survey. As can be seen in Table 1, the lottery winners and losers who participated in the study did not differ from each other very much in their demographic characteristics. The control group had slightly better educated mothers, but the difference was not significant, while those offered a scholarship were more likely to have mothers born outside of the United States. Those offered scholarships were more likely to receive Supplemental Security Income (SSI) for a family disability, while control group mothers were more likely to work outside of the home. All of these differences are modest and we can expect some significant differences to be produced by chance when comparing a large number of demographic characteristics. The overall picture is that despite non-participation in our study, we managed to preserve the similarity of the lottery winners and losers.

Table 1: Demographic Characteristics of Lottery Winners and Losers

Variable

Lottery Winners

Lottery Losers

Significance

Mother’s Education (11 point scale)

6.9

7.8

0.23

Mother U.S. Born

89%

96%

0.02

Attend Religious Services (5 point scale)

3.3

3.4

0.44

Receive Food Stamps

19%

19%

0.99

Receive Welfare

28%

24%

0.41

Receive Social Security

13%

11%

0.44

Receive Supplemental Security Income

28%

15%

0.01

Receive HUD Housing Vouchers

2%

5%

0.19

Family Income (from application)

$25,323

$25,297

0.97

Family Income (from survey)

$23,150

$24,800

0.14

Children in Household

2.4

2.2

0.12

Family Member in Jail

2%

1%

0.88

Student Has Physical Handicap

3%

1%

0.25

Student Has Learning Disability

9%

11%

0.55

Student is a Native English Speaker

97%

97%

0.77

African-American Mother

81%

80%

0.75

Two Parent Household

36%

33%

0.53

Male Student

49%

41%

0.1

Year of Mother’s Birth

1962

1963

0.28

Mother Employed Full-Time

60%

68%

0.03

Mother Single, Never Married

27%

29%

0.14

Mother is Baptist

38%

42%

0.96

N

206-267

135-161

 

Significance below .05 is conventionally considered statistically significant.

But we are not primarily interested in comparing outcomes of lottery winners to lottery losers. That is, our primary interest is in identifying the effect of using a scholarship to attend private school, not the effect of being offered a scholarship even if one does not use it. We therefore want to compare choice students to the other groups. As can be seen in Table 2, choice students differ from the other two groups of students (control and non-complying) whom we are calling “public” students. Even though some of the differences are statistically significant, the substantive differences are modest. The overall picture is of the choice students and comparison groups being quite similar, although clearly not identical.

Table 2: Demographic Characteristics of Choice and Public School Students

Variable

Choice

Public

Significance

Mother’s Education (11 point scale)

7.1

7.3

0.75

Mother U.S. Born

88%

94%

0.02

Attend Religious Services (5 point scale)

3.4

3.2

0.08

Receive Food Stamps

13%

22%

0.03

Receive Welfare

6%

9%

0.26

Receive Social Security

13%

12%

0.79

Receive Supplemental Security Income

7%

6%

0.87

Receive HUD Housing Vouchers

1%

5%

0.07

Family Income (from application)

$26,084

$24,714

0.24

Family Income (from survey)

$23,450

$23,850

0.88

Children in Household

2.4

2.2

0.12

Family Member in Jail

1%

2%

0.15

Student Has Physical Handicap

3%

2%

0.72

Student Has Learning Disability

4%

13%

0.00

Student is a Native English Speaker

98%

97%

0.45

African-American Mother

76%

85%

0.04

Two Parent Household

42%

29%

0.01

Male Student

49%

44%

0.34

Year of Mother’s Birth

1961

1963

0.01

Mother Employed Full-Time

51%

73%

0.00

Mother Single, Never Married

23%

31%

0.00

Mother is Baptist

33%

45%

0.00

N

145-189

197-239

 

Significance below .05 is conventionally considered statistically significant.

We employ two strategies in this study for comparing the outcomes of choice students to those of the other groups. The first strategy employs what is called a quasi-experimental research design in which observed differences between the groups that are theoretically expected to be related to the outcomes are controlled statistically.3  Because the groups are already very similar, we have less reason to fear that unobserved differences between the groups bias our estimates of the effect of using a scholarship. Concern about the unobserved differences between families that send their children to public and private schools has always limited our ability to draw conclusions from comparisons of the outcomes of students enrolled in public and private schools. Even after controlling for observed demographic differences, researchers could always wonder whether unobserved differences that were not being controlled statistically, such as parental motivation or the intellectual richness of home life, actually accounted for the differences in student outcomes instead of the schools.

In our case, however, the application process and lottery have produced groups for comparison that are already quite similar on observed as well as (in all likelihood) unobserved characteristics. All families had to be sufficiently motivated to complete an application for a scholarship. All families had to be low-income to qualify for a scholarship. A lottery was used to select who would be offered scholarships, creating, as we have confirmed, two groups that were nearly identical. While non-compliance and non-participation have caused the groups we are comparing to stray from being identical in their background characteristics, they are still quite similar so that controlling for observed characteristics is likely to produce results in which we can have high confidence.

The second strategy to identify the effect of using a scholarship is to use the lottery as an “instrument” to estimate who uses the scholarships that are offered.4  That is, we first predict who will use a scholarship, using whether someone won the lottery to help us make that prediction, and then we determine whether the students we predict used a scholarship have better outcomes. By using the predicted users of scholarships rather than the actual users, we remove the bias that may be introduced by the fact that the students who used the vouchers may differ (in unobserved ways) from the students who were offered a voucher but did not use them. Our estimated scholarship users will be nearly identical in their background characteristics to the groups against which we are comparing them. This technique, known as an instrumental analysis or a two-stage Heckman analysis, is a widely used strategy among economists that can produce very reliable findings.

Test Score Outcomes

Using these two strategies we can estimate the benefit of receiving a scholarship to attend a private school in Charlotte on student standardized test scores after one year. Using the quasi-experimental technique, we compute the effect of using a scholarship controlling for a host of background characteristics, including mother’s education, mother’s race, family income, two-parent household, and sex of student. These background characteristics are widely thought to be strongly related to student achievement in education research.5  We could control for additional background characteristics, but we would lose additional students from our analyses due to the fact that not all parents completed all questions on their surveys without gaining much explanatory power.

The benefit of receiving a scholarship on students’ math scores is 5.9 percentile points at the end of the first year (see Table 3). The benefit of using a scholarship to attend a private school on reading scores is 6.5 percentile points after one year. Gains in both math and reading are statistically significant at the conventional p < .05 level.

Table 3: The Effect of Attending a Private School with a Scholarship on Test Scores

The Effect of Attending a Private School with a Scholarship on Math Scores

 

Quasi-Experimental

Instrumental w/o background controls

Instrumental w/ background controls

Variable

Effect

Significance

Effect

Significance

Effect

Significance

Choice

5.9

0.04

6.1

0.01

6.2

0.10

African-American Mother

-13.5

0.00

 

 

-13.4

0.00

Mother’s Education

2.4

0.00

 

 

2.4

0.00

Family Income
(in $5,000 increments)

2.0

0.01

 

 

2.0.01 0

 

Two-Parent Household

2.8

0.40

 

 

2.6

0.44

Male Student

0.4

0.86

 

 

0.4

0.87

Non-Complying Student

0.7

0.85

 

 

 

 

Constant

10.3

0.09

29.1

0.00

10.3

0.10

N

357

 

436

 

357

 

Adjusted R-Square

0.14

 

0.01

 

0.14

 

The Effect of Attending a Private School with a Scholarship on Reading Scores

 

Quasi-Experimental

Instrumental w/o background controls

Instrumental w/ background controls

Variable

Effect

Significance

Effect

Significance

Effect

Significance

Choice

6.5

0.03

5.4

0.00

7.7

0.05

African-American Mother

-11.0

0.00

 

 

-10.7

0.00

Mother’s Education

2.8

0.00

 

 

2.7

0.00

Family Income
(in $5,000 increments)

1.6

0.06

 

 

1.7

0.05

Two-Parent Household

10.0

0.01

 

 

9.3

0.01

Male Student

-5.7

0.04

 

 

-5.7

0.04

Non-Complying Student

3.3

0.42

 

 

 

 

Constant

13.0

0.04

34.7

0.04

12.8

0.05

N

357

 

436

 

357

 

Adjusted R-Square

0.17

 

0.01

 

0.17

 

Significance below .05 is conventionally considered statistically significant.

When using the instrumental analysis it is arguable that it is not necessary to control for background characteristics because we have re-captured the nearly identical comparison groups produced by the lottery to award scholarships. The advantage of not controlling for any background characteristics is that we avoid losing any cases due to missing data from the parent surveys. An instrumental analysis without controlling for any background characteristics shows the benefit of using a scholarship to be 6.1 percentile points for math and 5.4 percentile points for reading. Both results are statistically significant.

The estimated effect of using a scholarship from the instrumental analysis increases somewhat if we add controls for background characteristics, although we do lose nearly 100 cases because of missing data on one or more variable. The benefit of receiving a scholarship on math scores in this analysis is 6.2 percentile points, while the benefit for reading is 7.7 percentile points. The math effect is statistically significant at p < .1 and the reading effect is significant at p < .05.

The test score results across these analyses are consistently positive and significant. Having access to a private school with a scholarship improves student performance on standardized test scores by between 5.4 and 7.7 percentile points for math and reading after only one year’s time. On average, a scholarship makes the difference between students scoring in the low 30s and the high 30s. This gain is fairly large. Using within sample variance, the benefit is approximately .25 standard deviations for math and reading, which education researchers generally consider large. To put the gain in perspective, the difference between minority and white students nationwide is approximately 1 standard deviation. The benefits observed from the Charlotte CSF program are roughly one-quarter as large at the end of the first year.

Parental and Student Satisfaction

Another important indicator of the benefit of a program on students is how parents describe those benefits. While parents’ judgments may be distorted by the desire to affirm their decision, parents are particularly well-positioned to assess effects on their own children given how much more contextual information they have about how their children are doing. According to parents, having a scholarship to attend private school is clearly beneficial. Parents were asked to assign their child’s school a letter grade, A through F. Nearly twice as many choice parents gave their child’s school an A (53%), compared to the public school parents (26%). (See Table 4) Choice parents were also much more likely to report being “very satisfied” with virtually all aspects of their children’s school: location, safety, teaching quality, course content, class size, facilities, student respect for teachers, information on student progress, religious observance, parental support for school, discipline, clarity of school goals, teamwork among staff, teaching moral values, academic quality, and teacher respect for students.

Table 4: Parental Satisfaction

Variable

Choice

Public

Significance

Would Give School an A

53%

26%

0.00

Percentage Very Satisfied With…

 

 

 

School Location

47%

29%

0.00

School Safety

58%

32%

0.00

Teaching Quality

54%

27%

0.00

What is Taught

64%

33%

0.00

Class Size

61%

24%

0.00

Facilities

53%

25%

0.00

Students Respect Teachers

61%

31%

0.00

Information on Student Progress

60%

29%

0.00

Observe Religion

65%

25%

0.00

Parental Support for School

58%

27%

0.00

Discipline

53%

30%

0.00

Clarity of School Goals

50%

25%

0.00

Teamwork Among Staff

54%

26%

0.00

Teaching Values

62%

27%

0.00

Academic Quality

55%

27%

0.00

Teachers Respect Students

58%

26%

0.00

N

185-190

231-242

 

Significance below .05 is conventionally considered statistically significant.

The older students who completed a survey during the testing sessions similarly reported significantly more positive assessments of their private school than did those students who did not receive a scholarship. Roughly two in five students would give their choice school an A compared to 32% of public school students. (See Table 5) When students were asked how they feel about going to school each day 24% of the public school students said that they did not want to go compared to 9% of choice students. And 24% of non-scholarship students agreed that they did not feel safe at school compared to 9% of choice students.

Table 5: Student Assessments of Schools

Variable

Choice

Public

Significance

Would Give School an A

40%

32%

0.05

Do Not Want to Go to School

9%

24%

0.00

Do Not Feel Safe at School

9%

24%

0.03

Strongly Agree that…

 

 

 

Teachers are Interested in Students

52%

28%

0.00

Teachers Listen

44%

26%

0.01

Teaches are Fair

35%

22%

0.00

Agree that…

 

 

 

Students Get Along with Teachers

66%

38%

0.00

N

96-98

107-109

 

Significance below .05 is conventionally considered statistically significant.

Parental reports confirm student perceptions about safety at school. More than a third of public school parents reported problems with fighting in school (36%) compared to 16% of choice parents. (See Table 6) One-quarter of public school parents reported problems with racial conflict compared to 12% of choice parents. 22% of public school parents reported problems with guns or weapons at their children’s elementary schools compared to 11% of choice parents. And 25% of public school parents reported problems with destruction of property at school compared to 12% of choice parents.

Table 6: Parent Description of School:
Percentage Reporting Problems are Somewhat or Very Serious

Variable

Choice

Public

Significance

Fighting

16%

36%

0.00

Racial Conflict

12%

25%

0.00

Guns or Weapons at School

11%

22%

0.00

Destroying Property

12%

25%

0.00

Cheating

16%

36%

0.00

Cutting Classes

15%

26%

0.01

Tardiness

23%

33%

0.08

N

185-188

233-238

 

Significance below .05 is conventionally considered statistically significant.

School Facilities and Services

Given the overwhelmingly positive description of the choice schools and given the test score improvements, one might expect that the private schools are simply more luxurious schools with better resources. Far from it. Most of the private schools at which students used scholarships operate on nearly half as much money per pupil as do the public schools. Tuition at most of the private schools is well below $3,000 and additional fundraising brings no more than a few hundred dollars per student.

With far less money it is not surprising to discover that the private schools have more sparse facilities and fewer services to offer. For example, only 70% of choice parents described their school as having a library compared to 90% of the public school parents. (See Table 7) Only 63% of choice parents said that their school had a gym compared to 91% of public school parents. Only 71% of choice parents said that their school had a cafeteria compared to 89% of public school parents. Parents also reported fewer school services at the private schools. Only 18% of choice parents said that their school had a program for students learning to speak English compared to 50% of public school parents. Only 49% of choice parents said that their school had a program for learning disabilities compared to 71% of public school p arents. Only 51% of choice parents reported program for gifted students at their schools compared to 72% of public school parents. Choice parents were also less likely to report that their school had a counselor, nurse, music program, art program, and prepared lunches.

Table 7: Parent Description of School Facilities and Services

Variable

Choice

Public

Significance

Computer Lab

80%

85%

0.23

Library

67%

90%

0.00

Gym

63%

91%

0.00

Cafeteria

71%

89%

0.00

Program for Non-English Speakers

18%

50%

0.00

Individual Tutors

64%

64%

0.92

Program for Learning Disabilities

49%

71%

0.00

Program for Gifted Students

51%

72%

0.00

School Counselor

66%

83%

0.00

Nurse

46%

79%

0.00

Music Program

85%

93%

0.01

Art Program

68%

79%

0.02

After-School Program

92%

83%

0.00

Prepared Lunch

74%

92%

0.00

N

125-185

130-240

 

Significance below .05 is conventionally considered statistically significant.

There were some things that were equally or more available at choice schools. For example, choice and public schools were roughly equally likely to have a computer lab. And choice schools were equally likely to offer individual tutors and more likely to offer after-school programs. When parents report that they are more satisfied with the choice school facilities, they clearly must be focusing on these features that they believe are more important. Choice schools appear to have far fewer resources but to concentrate those resources on providing the facilities and services that parents value most.

What Might Account for Choice School Success?

If the private schools are not better funded and do not have nicer facilities and services by objective standards, why do parents like them so much? The most obvious answer is that parents like the choice schools because their children are learning more. But what might account for this better student achievement? While this study is not designed to address this question fully, it is possible to speculate based on the evidence that was collected. Some of the most important differences between the choice and public schools pertain to the quality and motivation of teachers in the two sectors. As we have already seen, parents give very strong marks to the quality of instruction at the choice schools. Interestingly, so do the students. Students are almost twice as likely to report that teachers at choice schools are “interested in students” than are public school students. (See Table 5) Choice students are also significantly more likely to report that their teachers listen to students, that teachers are fair, and that students get along with teachers.

Despite having less money for teacher salaries and benefits, private schools appear to be better able to recruit quality teachers and dismiss bad ones. They may attract more quality teachers because they can offer positive working conditions, an organization with a clear sense of mission, and greater autonomy in the classroom. Layers of bureaucratic regulations and control in the public schools, perhaps a by-product of political governance of the schools, makes it difficult for public schools to maintain positive working conditions, agree on a clear mission, or provide autonomy in the classroom. Importantly, school district and union rules also make the removal of bad teachers much more difficult in public schools than in private schools.

Choice and public schools also differ in their overall size and in their average class size. The median choice student is enrolled in a school that has between 151 and 300 students. The median public school student is in a school that has between 451 and 600 students. The median choice student is in a class that has between 11 and 15 students, while the median public school student is in a class with between 21 and 25 students. Education researchers are increasingly recognizing that there may be diseconomies of scale in education.6  That is, smaller school districts tend to do better than larger school districts, smaller school buildings tend to do better than larger school buildings, and smaller classrooms may do better than larger classrooms. Smallness may permit the development of a sense of community and common purpose, which may be key to school success. And smallness obviates the need for rigid rules that restrict the autonomy of principals and teachers.

Some critics of school choice suggest that small classes in private schools “explain” the achievement benefits of voucher and private scholarship programs. If only public schools were provided with additional resources to reduce class size, they too could improve achievement. This, of course, begs the question: why have the private schools with fewer resources been able to produce significantly smaller classes than public schools? And what assurance is there that additional funds for public schools will lead to reduced class size and not to higher paid teachers or more non-teaching staff?

Interestingly, adding class size to the model in Table 3 that estimates student achievement shows that class size is not significantly related to student achievement in our sample. In other words, class size does not “explain” the achievement benefits of receiving a scholarship to attend private school in Charlotte.

In addition, one should not attempt to explain why private schools appear to outperform public schools while attempting to estimate whether private schools outperform public schools. By analogy, if we want to know whether the Cubs or Yankees are better baseball teams, we should not control for pitching, hitting, and fielding. Pitching, hitting, and fielding may help explain why one team is better than another, but they should not be considered when assessing whether one team is better than another. Similarly, when we are addressing whether students do better when they have access to a private school with a scholarship we should not attempt to control for those factors that may help explain why private schools may be better.

Creaming and Dumping

Another prominent explanation for private school success is that private schools are able to select their students by skimming off the cream of students and dumping the undesirable students. In truth, public schools can also be selective. Some magnet and other public school programs have academic or racial criteria for admission. And students whom the public schools decide they cannot educate properly are sometimes sent to other public schools or to private schools at public expense. Not every public school is obligated to accept every student.

In our sample we saw little evidence to suggest that private schools were creaming the best students and dumping the worst. First, almost no private schools were administering admissions tests to select academically advantaged students. Families who were unable to get their children into the schools they desired were asked to provide the reasons for their inability to gain access to those schools. More than three-fifths of these families cited financial constraints as blocking their access to a desired school. According to parental reports only two students out of all of the students offered a scholarship failed to gain admission to a private school because of an admissions test.

Second, there is no evidence that private schools expelled undesirable students or asked them not to return. Parents of students who did not complete the year at the same private school were asked to describe the reason for their switch. Not one reported that they switched schools because their child was expelled. And of those parents who reported that they might not return to the same school next year not one reported that their child was asked not to return. In short, there is virtually no evidence that the choice schools academically screened their students for admission or expelled or “counseled out” students they found undesirable.

Parents were also asked whether their children had any physical handicaps, learning disabilities, or issues learning to speak English. Very few reported physical handicaps, only 3% of choice students and 2% of public school parents. Similarly low percentages of choice and public school parents reported that English was not their child’s native language. However, choice parents reported fewer children with learning disabilities (4%) than did public school parents (13%). As we have already observed, given their lower level of funding fewer private schools offer special programs for learning disabilities. This difference in the percentage of students with learning disabilities may also be partially explained by differing incentives in the public and private schools to label students as having learning disabilities. Public schools obtain additional resources for students labeled as learning-disabled and may be able to exempt learning-disabled students from accountability testing.

While this difference in learning disabled students at choice and public schools is significant, it is not necessarily evidence of creaming or dumping. It may be evidence of parental choice. Parents of children with special needs are more likely to choose schools that have additional funds to offer programs that address those special needs. A fair test of whether private schools are avoiding learning disabled students would compare the rates of learning disabilities when private schools are given the same additional resources to serve those children as the public schools receive. In the absence of such a test, this evidence on learning disabilities is ambiguous.

In general, choice schools were accepting students with scholarships who were considerably more disadvantaged than typical students in Charlotte. Three-quarters of the choice students were African-American, while a little more than one-third of all students in the Charlotte-Mecklenburg school district are African-American.7  As of 1990 the average family income in Charlotte was nearly $34,000, almost $10,000 more than the average family income of choice students 10 years later. Almost one-third (32%) of choice families report that they receive some kind of public assistance, such as food stamps or welfare, while the 1990 census reports that only 5% of households in Charlotte were on public assistance. And even after one year of the scholarship, choice students were still scoring well below the national average on standardized tests (although they were scoring significantly better than they would have had they not received the scholarship).

It takes some doing to suggest that the scholarship families that enrolled in private school are the cream when those families are more likely to be African-American, low income, on public assistance, and score below-average on test scores than typical families in Charlotte or the United States. It is clear that the CSF program in Charlotte is successfully targeting disadvantaged students. While it may not reach the most severely disadvantaged, just as Food Stamps or housing vouchers do not always reach the most disadvantaged, the scholarship program is clearly offering opportunities to families that lack them. And it is also clear that the private schools are taking on these disadvantaged students, not creaming off the best and dumping the worst.

Implications for School Choice Policies

The privately-funded scholarship program in Charlotte differs from what a publicly-funded school choice program would likely be in a number of respects. First, the scholarship had a low monetary value and always required a significant co-payment from the family toward tuition. A publicly-funded voucher would likely be worth considerably more money and would require little if any co-payment from the receiving families. This difference may alter the benefit we would expect to see from gaining access to private schools. The additional money a publicly-funded voucher would provide to private schools might increase the expected benefit, but the reduced co-payment from families might alter the characteristics of participating families and reduce the benefit.

Second, privately-funded scholarships place little or no regulation on the activities of private schools, while publicly-funded vouchers would likely carry with them more regulation. That regulation might improve the benefits of the program by ensuring equal access and the provision of consumer information, but regulation might also reduce the benefits of the program by encumbering schools.

Third, the CSF scholarship program was small enough so that its recipients could be accommodated by spare capacity in existing private schools. A larger, publicly-funded school choice program would require the addition of new private schools. The outcomes in new private schools might be better or worse than that observed in existing schools.

There is no way of addressing these issues fully without attempting additional publicly-funded programs on a larger scale. The results from the evaluation of the Charlotte CSF scholarship program strongly suggest that attempting larger-scale, publicly-funded programs is desirable. The positive findings from Charlotte are consistent with positive results from evaluations of privately-funded programs in New York, Washington, D.C., and Dayton as well as pilot, public choice programs in Milwaukee and Cleveland. Whether those positive results will hold when school choice is attempted in a more complete way cannot be known at present. The existing evidence is encouraging enough that we should implement new school choice programs to see if these significant benefits can be reproduced on a larger scale.

Notes

  1. The author would like to thank all of the people at the John Locke Foundation who contributed their time to this study. In particular, Sherri Joyner, Kory Swanson, Linda Hunt Williams, and John Hood devoted significant time, energy, and foundation resources to make this study possible. Rob Fusco and Chris Hammons provided valuable research assistance. Henry Olsen and Michael Barreiro of the Manhattan Institute provided important administrative support and encouragement. Most importantly, the author would like to thank the schools, parents, and students in Charlotte for their participation.
  2. See Jay P. Greene, “A Survey of Results from Voucher Experiments: Where We Are and What We Know,” Civic Report, The Manhattan Institute for Policy Research, Number 11, July 2000. Available on-line at: http://www.manhattan-institute.org/html/cr_11.htm.
  3. See Thomas D. Cook and Donald T. Campbell, Quasi-Experimentation: Design and Analysis Issues for Field Settings (Boston: Houghton Mifflin) 1979.
  4. See Jacob and Patricia Cohen, Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences, Second Edition (Hillsdale: Lawrence Erlbaum) 1983.
  5. See for example, Barbara Schneider and James S. Coleman, eds., Parents, Their Children, and Schools (Boulder: Westview) 1983.
  6. See Gary Burtless, ed., Does Money Matter (Washington, D.C.: Brookings) 1996.
  7. Demographic characteristics of the Charlotte-Mecklenburg school district were obtained from the School District Data Book Profiles, available on the web at: http://govinfo.library.orst.edu/sddb-stateis.html, accessed on August 15, 2000.

 


Center for Civic Innovation.

EMAIL THIS | PRINTER FRIENDLY

CR 12 PDF

WHAT THE PRESS SAID:

Low-income students make academic gains by Jay P. Greene, The Charlotte Observer, September 22, 2000
Privately Financed Vouchers Help Black Students, Two Studies Find by Darcia Harris Bowman, Education Week, September 6, 2000
Private school lottery winners score higher: Poor children subsidized by fund by Celeste Smith, Charlotte Observer, August 31, 2000
Journal: Vouchers raise the bar The Atlanta Journal-Constitution, August 30, 2000
Vouching for vouchers by Jacob Sullum, townhall.com, August 29, 2000

SUMMARY:
This important study finds that children in Charlotte who attended private schools with the aid of tuition scholarships showed significant improvement in their scores on standardized reading and math tests. The study also finds that parents of scholarship children rate their school much higher than do parents whose children attend Charlotte public schools.

TABLE OF CONTENTS:

Executive Summary

Introduction

Research Design

Comparability of Groups

Table 1: Demographic Characteristics of Lottery Winners and Losers

Table 2: Demographic Characteristics of Choice and Public School Students

Test Score Outcomes

Table 3: The Effect of Attending a Private School with a Scholarship on Test Scores

Parental and Student Satisfaction

Table 4: Parental Satisfaction

Table 5: Student Assessments of Schools

School Facilities and Services

Table 6: Parent Description of School Percentage Reporting Problems are Somewhat or Very Serious

What Might Account for Choice School Success?

Table 7: Parent Description of School Facilities and Services

Creaming and Dumping

Implications for School Choice Policies

Notes

 


Home | About MI | Scholars | Publications | Books | Links | Contact MI
City Journal | CAU | CCI | CEPE | CLP | CMP | CRD | ECNY
Thank you for visiting us.
To receive a General Information Packet, please email support@manhattan-institute.org
and include your name and address in your e-mail message.
Copyright © 2009 Manhattan Institute for Policy Research, Inc. All rights reserved.
52 Vanderbilt Avenue, New York, N.Y. 10017
phone (212) 599-7000 / fax (212) 599-3494