NAEP 2009 Math Results Show Little Progress for NYC Schools

Last week, math scores for 2009 were released for the eighteen cities now participating in the NAEP Grade 4 and 8 Trial Urban District Assessment (TUDA) component, released bi-annually as “The Nation’s Report Card.” Operated under the aegis of the U.S. Department of Education and reported by their National Center for Educational Statistics, the NAEP’s are the only math and reading exams administered across the fifty states (reading test results are expected in Spring 2010). Given on a statistical sampling basis without high stakes consequences, these exams are generally considered the single best measure of national educational progress.

The Math results for 2009 were not the best of news for NYC. Among the eleven large cities who participated in 2007, only Boston and D.C. showed statistically significant gains in Grade 4 and only Austin and San Diego in Grade 8. No city declined, but the rest, including NYC, showed either no gains or gains sufficiently modest that they couldn’t reliably be attributed to real gains (as opposed to random chance). For NYC, these results stand in sharp contrast to the rather sizable gains in Grade 4 and 8 Math reported over the last two years via the NYS Grade 3-8 annual exams (on which School Progress Reports, grade retention, principals’ bonuses, and the Mayor’s political persona are based).

As reported in the local media (Jennifer Medina in the NY Times, Rachel Monohan and Meredith Kolodner in the NY Daily News, Yoav Gonen in the NY Post, and Beth Fertig at WNYC.org), NYC showed a one-point gain in average exam score in Grade 4 (from 236 in 2007 to 237 in 2009) and a three-point gain (from 270 to 273) in Grade 8. In addition, the percentage of Grade 4 students judged Proficient or better on the NAEP exam rose from 34% in 2007 to 35% in 2009; for Grade 8, the numbers were 22% in 2007 to 26% this year. By comparison, the NYS exams showed NYC Grade 4 proficiency or better increasing from 74.1% to 84.9% and Grade 8 moving up an astonishing 25 percentage points, from 45.6% to 71.3%. Comparable numbers since 2003 are shown below.

Grade 4 - % Proficient or Higher (Level 3+4 for NYS exams, Proficient + Advanced for NAEP)

2003 --- NYS 66.7% --- NAEP 21%

2005 --- NYS 77.4% --- NAEP 26%

2007 --- NYS 74.1% --- NAEP 34%

2009 --- NYS 84.9% --- NAEP 35%

Grade 8 - & Proficient or Higher (Level 3+4 for NYS exams, Proficient + Advanced for NAEP)

2003 --- NYS 34.4% --- NAEP 21%

2005 --- NYS 40.8% --- NAEP 21%

2007 --- NYS 45.6% --- NAEP 22%

2009 --- NYS 71.3% --- NAEP 26%

The inflation in all the NYS-based numbers naturally suggests that NYS/NYC students’ mathematical knowledge is nowhere near what is claimed by NYSED and NYCDOE.

This hardly comes as a surprise to most; it largely reflects the inherent fallacy of the NYSED’s testing regime, whose high stakes under NCLB have been substantially amplified under mayoral control of NYC schools until test prep and teaching to a specific test have come to dominate math teaching in the city. Nevertheless, measuring NYC’ students’ mathematical progress against the NAEP’s does suggest that, at least at Grade 4, both sets of scores are moving upward in some degree of encouraging lockstop. The picture in Grade 8 is far different and far more gloomy, particularly viewed within the framework of the huge reported gains in 2008 (14 percentage points, to 59.6%) and 2009 (another 11.7 percentage points, to 71.3% as shown in the table above).

These latest NAEP results offer a highly reliable reality check on the claimed math gains of NYC public school students. At Grade 4, the NAEPs suggest that the NYS test results have a degree of validity with regard to progress, although the 50-percentage-point gap on achievement is a major contradictory indicator of mastery level. At Grade 8, the NAEP’s paint a truly worrisome picture of a 45-percentage-point achievement gap and almost no measured progress in six years on the NAEPs during a period when NYC results on State exams signal gains bordering suspiciously on miraculous.

More disconcerting still is the realization that the 2009-year Grade 8 students who took these latest NAEPs are the first "pre-high-school end products" of the NYC public school system under Joel Klein and mayoral control during the six-year period (2003 - 2009) since the TUDAs and mayoral control both began.

Michael Bloomberg’s campaign for his third mayoral term was predicated in no small part on his claims to be NYC’s “education mayor” as justified by NYS exam scores and a host of other DOE self-generated statistical measures. In my next posting, I will delve much more deeply into the NAEP’s raw data to compare NYC’s progress since 2003 against the nine other large cities who have participated in TUDA since that time and also reveal how many of the DOE’s (and Mayor’s) claims are simply not supported by the only truly independent measure of NYC public school education currently available.