So what's the story? Have NYC schools made progress in test scores, or not?

Today's Daily News reports on how NYC parents are reacting in the Bronx and in Manhattan to the revelation that the great progress claimed by Bloomberg and Klein in the past was a fraud: "The scores make you want to cry," said Monica Major, a Bronx parent and president of School District 11's Community Education Council. "For years, we've been bamboozled and lied to."

Or as as Diane Ravitch writes in an oped, "When the [state] scores were released, there was a sound of bursting bubbles across the state. What once were miracles turned into mirages.”

Yet in recent days, Bloomberg and Klein have tried to argue that the evidence is still strong that city students have made "dramatic progress" under their leadership.

First, they have repeatedly claimed that since the city's average scores on the state exams have been rising over time, the fact that the proficiency cut scores have now been drastically lowered is irrelevant.

Yet the city's average scores on the state exams are likely just as unreliable as its proficiency levels, since the questions on the test have gotten easier over time.

See an earlier posting, in which Steve Koss discusses a 2007 Daily News expose showing how the increase in "P" (or "probability) values on the field test questions indicated that the state tests had become simpler between 2002-2005. (The chart to the right was originally in the Daily News.)

To prove this, intrepid reporter Erin Einhorn even gathered a bunch of children to take both the 2002 and 2005 math exams, and out our 34 kids, 24 of them did better on the 2005 exam.

As one of them said, "the 2002 questions were more complicated than in 2005... In 2005, they kept it short, simple and sweet.”

Really, the only semi-reliable measure we have of student achievement over this period is the city's performance on the national exams known as the NAEPs. Klein has also made questionable claims in recent days in relation to the NAEPs.

Here is what he said to the Wall St. Journal: "In both fourth-grade math and reading, New York City now performs at the same level as the nation overall," the schools chancellor said."

Is this true? No. In no subject or grade does the achievement level of city students match that of the nation as a whole, including 4th grade reading or math; though the gaps are smaller in that grade than in 8th.

A more meaningful comparison would be to look at in the change in NAEP scores compared to other large cities from 2003-2009, to see whether there has been exceptional progress during the Bloomberg/Klein era .

First I looked at the change in the percent of students at or above basic over this period, in NYC compared to other large cities and to Atlanta, which has made exemplary progress.

Note that compared to other large cities, NYC has done a bit better on average in two categories and worse in two: 8th grade reading and 8th grade math.

In all subjects, however, NYC's gains have been far less than those of Atlanta, (which incidentally has neither mayoral control nor has it been awarded the Broad prize.)

I also analyzed NYC's record in 8th grade reading, to see how our schools have done compared to other large urban districts.

This is what I found: NYC comes in dead last among the ten cities tested since 2003 in this subject -- the only city to make absolutely no progress.

Then I looked at the achievement gap issue between racial and ethnic groups, which Bloomberg and Klein's repeatedly claimed to have narrowed over the past few years. At one point, Bloomberg even went before Congress and boasted that he had cut it in half. Is that true?



No, of course not; In NYC schools, according to the NAEPs, the achievement gap is still very substantial, in all subjects and grades, from 23 to 35 points depending on the category.

But have these gaps narrowed over time? Well, the black-white gap has shrunk by one or two points in 4th grade reading and 8th grade math, which is relatively small in relation to the overall size of the gap; and the gap has increased slightly in the other two categories.

In terms of the white-Hispanic gap, Bloomberg and Klein's record is even worse. In three out of four categories, the gap has increased; and it has grown much larger in 8th grade reading and math.

So, when you look at the NAEP scores, the progress made in NYC under Bloomberg and Klein compared to other large cities is only a little better in some areas, and in some areas, like 8th grade reading or the Hispanic achievement gap, far worse.