Before we begin, some definitions:
- DCPS Schools are the traditional DC public schools.
- Public charter schools are schools who were granted their charter by the DC Public Charter School Board.
- BoE charter schools are schools who were granted their charter by the Board of Education.
- Private schools are not included.
The first set of statistics we will look at is the 2004 NCLB Adequate Yearly Progress (AYP) metrics. Note: this is the latest full set of data I have available. These are calculated by looking at the students' results of the Stanford 9 tests. The results are broken out by elementary and secondary schools, and into separate math and reading scores. Each score shows the percentage of students who tested as "proficient" for their grade level.
CORRECTION: Corrected the All Schools: Secondary Math number. It 39.32%, not 39.52%.
As you can see, at the elementary level DCPS and public charter schools are roughly equivalent for both math and reading scores, with BoE charter schools lagging (although I'm unsure how statistically significant the differences are between the three populations; this will be part of further analysis). When we get to the secondary level however, the numbers begin to spread out. All three school populations see reductions in proficiencies from their elementary school metrics, with public charter schools seeing the least amount of erosion: 9% in reading, and 2% in math. DCPS loses 15% in reading, and 20% in math. Again, we'll have do more analysis to determine the statistical significance of these differences, but at first glance the losses look particularly painful for the DCPS schools. Furthermore, considering that the BoE schools' elementary numbers were not very robust to begin with, that 13% drop in both sets of metrics really hurts.
CORRECTION: BoE charter schools lost 13% in both math and reading from elementary to secondary school, not 15% as originally written.
So, what does this all mean? Well, at first glance, DCPS and public charter elementary school students perform at roughly the same level in both reading and math . However, while both see subsequent erosion at the secondary school level, on average students at public charter schools outperform DCPS students, especially in math. Finally, students at BoE charter schools lag students at both DCPS and public charter schools, and this is observable at both the elementary and secondary levels.
To caution, this is very preliminary data, and I'll continue to crunch the numbers on this end. If I (or you!) find any errors in the analysis, I'll update this posting, documenting what has changed and why. As I said above, if you have a hankering to play with numbers, go ahead and check out the data for yourself.
UPDATE: I have done more number crunching on this data set. First, the population samples for each data segment:
CORRECTION: The PCS secondary math number is 3,991, not 9,991 as previously listed. This was a transcribing error, the analysis below was done with the correct data.
I then tested that to see if the test scores for each population were significantly (statistically) different. (See note below for methodology and caveat)
- There is no significant difference between the level of proficiency from the elementary reading tests of DCPS and public charter schools. There is most likely a difference between the levels of proficiency from the elementary reading tests of the BoE charter schools and the other schools; the statistical tests were persuasive (p<.10) but not where we would want them to be certain (p>.05)
- There is no significant difference between the level of proficiency from the elementary math tests for DCPS and public charter schools. The level of proficiency from the elementary math tests of the BoE charter schools are, however, significantly lower than the proficiency of the other two groups of schools.
- At the secondary school level the difference between both the reading and math proficiency for each of the three school systems is significant: i.e. we can say that in both the math and reading tests that the public charter school students were more proficient than DCPS students, who in turn were more proficient than BoE charter students.
One thing I want to stress is that this is an analysis at the school system level; this does not imply that any public charter high school had higher proficiency than any DCPS high school. Within each school system are schools with high proficiency rates and schools with low proficiency rates. This analysis simply looked at the entire test taking population of each school system. I do intend to do some school-by-school analysis in the future, but it was beyond the scope of this particular exercise.
That's it for tonight and this post, baring any data or statistical screw-ups anyone finds. Next up: similar analysis of the 2003 data and 2005 data when available, and a year-to-year comparison for each school system.
UPDATE 3: Commenter Ed Researcher, who does this sort of thing for a living, makes some good points in the comment section that are worth reading.
UPDATE 2: Here's an interesting tidbit. According to the help page for the AYP reports the Number in Group metric is
Number of students enrolled in the group for a ‘Full Academic Year’If you add up the elementary and secondary school Numbers in Group for DCPS and BoE charter schools you get a grand total of 41,823 students. In 2003 the official enrollment number published by DCPS was 65,099 students. Even if you don't count the alternative programs, special education, and tuition grant students (I have no idea what their testing responsibility is) that still leaves over 60,000 students. I assume that the 2004 enrollment numbers aren't that far off of 2003 (if anyone has them let me know and I'll update this), so the question is, where did the other ~20K students go? Why were they not tested? Does anyone have a better understanding regarding who does and does not get tested, i.e. am I missing something?
UPDATE 2a: I did miss something. DCPS only tests 3rd, 5th, 8th and 10th grade, which explains why the test taking population was less than the entire school population.
WARNING: I've been plugging away with statistics for about 15 years, but its been a donkey's age since I did much work that went beyond your bog-standard gaussian distributed data-sets. I'm a little rusty with statistics around categorical data, so if anyone notices that I've done something horribly wrong please drop me a line in the comments.
The data presented for the Districts public and charter schools only shows how many people took the test and what percentage of them passed it. What we don't know is how well or badly each individual test taker did. For example, if 60% was the passing grade for a test and it shows that 100 students passed the test we don't know if those 100 students were brilliant and had perfect scores, they all just scraped by with 60s or something in-between. All we know is that they either passed or failed. Thus, for each population we can only back out the number of students who passed and the number of students who failed (in NCLB lingo, students who were proficient vs. students who were not). So I used Fisher's Exact Test to compare the pass/fail rates between the various school systems.
Greetings Carnivalites. Feel free to make yourself at home and read all about DC education. There is more coverage of NCLB metrics here and here.