Pre-Rhee and Post-Rhee

For those of you who believe that Michelle Rhee’s resignation today will hurt the achievement gains experienced by the DC school system; take a look at these charts (click to enlarge), with DC NAEP scores from 2003-2009.

It appears that DC schools were improving in terms of test scores at about the same rate pre-Rhee (before 2007) than after she got there, in every subject except 8th grade reading, where the improvement was greater before. (The source for these charts is the Rutgers report released today on mayoral control ).

Of course how much of this is due to her or her predecessors' educational reforms and how much to other factors (eg gentrification) is hard to discern. And many of Rhee's policies, such as IMPACT, the new teacher evaluation system, really won’t have show an effect until the next few years, if they are ever fully implemented.

Listen up, mayor and the NY Times

The Times ran an editorial last week, "Parents need to know", which claimed that NYC parents didn't understand how the weak state exams had been "shortchanging students" and that they did their "children no favor" when they disrupted a Panel for Education Policy meeting on Monday. This was after the PEP chair, David Chang, refused to let them speak in response to a long, deceptive power point that claimed, despite the collapse of the state test score bubble collapse, that the schools had made great progress.

The editorial also excused the state test score inflation by stating, "Weak state tests are a chronic problem throughout the country — one that education departments are only beginning to come to grips with." Oh please. The NY test score inflation has been obvious to nearly all objective observers since at least 2007, despite the fact that it conveniently allowed the mayor to claim great improvements during his campaigns for the renewal of mayoral control and re-election, illusions that were bought hook line and sinker by the mayor's allies on the Times and the other editorial boards.

The editorial ended by claiming that the schools have nevertheless been "narrowing the performance gap between white and minority students." Yet the gold standard, the national exams called the NAEPs, show no narrowing of the achievement gap in any grade or subject since the Klein regime began. Truly, the Times editors "need to know" and start living in the real world, for they, along with the chancellor and mayor, have lost all credibility, and should stop criticizing parents while displaying their ignorance of what's really going on in our schools.Our mayor, who famously said people could "boo him at parades" if people didn't like his educational policies, will likely see more booing of his Panel of Eight Puppets in the months to come, unless he wakes up and starts to listen.

So what's the story? Have NYC schools made progress in test scores, or not?

Today's Daily News reports on how NYC parents are reacting in the Bronx and in Manhattan to the revelation that the great progress claimed by Bloomberg and Klein in the past was a fraud: "The scores make you want to cry," said Monica Major, a Bronx parent and president of School District 11's Community Education Council. "For years, we've been bamboozled and lied to."

Or as as Diane Ravitch writes in an oped, "When the [state] scores were released, there was a sound of bursting bubbles across the state. What once were miracles turned into mirages.”

Yet in recent days, Bloomberg and Klein have tried to argue that the evidence is still strong that city students have made "dramatic progress" under their leadership.

First, they have repeatedly claimed that since the city's average scores on the state exams have been rising over time, the fact that the proficiency cut scores have now been drastically lowered is irrelevant.

Yet the city's average scores on the state exams are likely just as unreliable as its proficiency levels, since the questions on the test have gotten easier over time.

See an earlier posting, in which Steve Koss discusses a 2007 Daily News expose showing how the increase in "P" (or "probability) values on the field test questions indicated that the state tests had become simpler between 2002-2005. (The chart to the right was originally in the Daily News.)

To prove this, intrepid reporter Erin Einhorn even gathered a bunch of children to take both the 2002 and 2005 math exams, and out our 34 kids, 24 of them did better on the 2005 exam.

As one of them said, "the 2002 questions were more complicated than in 2005... In 2005, they kept it short, simple and sweet.”

Really, the only semi-reliable measure we have of student achievement over this period is the city's performance on the national exams known as the NAEPs. Klein has also made questionable claims in recent days in relation to the NAEPs.

Here is what he said to the Wall St. Journal: "In both fourth-grade math and reading, New York City now performs at the same level as the nation overall," the schools chancellor said."

Is this true? No. In no subject or grade does the achievement level of city students match that of the nation as a whole, including 4th grade reading or math; though the gaps are smaller in that grade than in 8th.

A more meaningful comparison would be to look at in the change in NAEP scores compared to other large cities from 2003-2009, to see whether there has been exceptional progress during the Bloomberg/Klein era .

First I looked at the change in the percent of students at or above basic over this period, in NYC compared to other large cities and to Atlanta, which has made exemplary progress.

Note that compared to other large cities, NYC has done a bit better on average in two categories and worse in two: 8th grade reading and 8th grade math.

In all subjects, however, NYC's gains have been far less than those of Atlanta, (which incidentally has neither mayoral control nor has it been awarded the Broad prize.)

I also analyzed NYC's record in 8th grade reading, to see how our schools have done compared to other large urban districts.

This is what I found: NYC comes in dead last among the ten cities tested since 2003 in this subject -- the only city to make absolutely no progress.

Then I looked at the achievement gap issue between racial and ethnic groups, which Bloomberg and Klein's repeatedly claimed to have narrowed over the past few years. At one point, Bloomberg even went before Congress and boasted that he had cut it in half. Is that true?



No, of course not; In NYC schools, according to the NAEPs, the achievement gap is still very substantial, in all subjects and grades, from 23 to 35 points depending on the category.

But have these gaps narrowed over time? Well, the black-white gap has shrunk by one or two points in 4th grade reading and 8th grade math, which is relatively small in relation to the overall size of the gap; and the gap has increased slightly in the other two categories.

In terms of the white-Hispanic gap, Bloomberg and Klein's record is even worse. In three out of four categories, the gap has increased; and it has grown much larger in 8th grade reading and math.

So, when you look at the NAEP scores, the progress made in NYC under Bloomberg and Klein compared to other large cities is only a little better in some areas, and in some areas, like 8th grade reading or the Hispanic achievement gap, far worse.

Bloomberg and Klein's personal credit recovery program

See the presentation given by Klein to business leaders yesterday. Some news accounts here: Times, GothamSchools, NY1, Post, Daily News.

As usual, the Chancellor tried to take credit where no credit is due -- for big increases in test scores that occurred in the winter and spring of 2003, despite the fact that his policies were not implemented until the following fall. Perhaps these retroactive claims are Klein and Bloomberg’s personal credit recovery program.

In fact, DOE officials and the mayor were quite subdued when there were big test score gains in 2003. For more on this, see our blog here.

As the Times pointed out when the state ELA scores were first reported on May 21, 2003:

“The city's positive results come at a time when Mayor Michael R. Bloomberg and his schools chancellor, Joel I. Klein, are trying to overhaul the public school system and impose a uniform reading and math curriculum at all but the highest performing schools.

…Experts said the sharp increase in test scores could prove problematic for Mr. Bloomberg and Mr. Klein, since it is too early for them to take credit and sets a benchmark for next year that may be hard to match….

The higher scores, particularly at schools that have showed sustained increases since 1999, also gave new ammunition to critics of Mr. Bloomberg's changes, who said that they might do more harm than good by disrupting existing programs….

This is how the state math gains were reported when they were released on October 22, 2003:

“Fourth graders across the state made stunning gains in their math scores last spring, with even sharper increases in New York City…In the city, news of the gains, which were particularly pronounced in the Bronx and in some of the poorest-performing districts, elicited cheers among teachers and principals. But not everyone greeted the news so enthusiastically:

The suggestion that city schools were on the upswing put Chancellor Joel I. Klein, who is overhauling them, in a tricky position. While the chancellor’s critics pounced upon the higher scores as evidence that the school system did not need such an overhaul, some of his allies acknowledged that he would now be under even more pressure to show gains next spring.

A more accurate and fair analysis would look at data only from 2003-2008; instead of claiming credit for gains that occurred before the Bloomberg/Klein policies were put into place.

2.. None of these gains is meaningful given the evidence of the NAEPs, which shows no significant progress in NYC since 2003 in any grade or subject except for 4th grade math. We know that the state tests have been inflated, and there’s been terrific amount of test prep in NYC and teaching to the test, unrelated to any real gains in learning.

3. See slide 13: as of 2004, still only 55% of NYC eighth graders who make a 700 --level 3 – graduate in four years. (not even counting the discharge rates which would bring their actual graduation rate even lower. ) This is a huge repudiation of the quality of our high schools. Even the fact that only 87% of 8th grade students who hit level four, the highest level, graduate on time is pretty awful.

4.No info is offered on the results of the science and social studies exams – which are low-stakes tests, like NAEPs, and thus more reliable. Unlike math and ELA scores, for which the DOE offers financial rewards to teachers, schools, and kids, with little or no protections against cheating, these are the exams where we could expect less distortion in results.

NYC continues to have some of the lowest scores in the state in those subjects, even in the largely middle class districts. 28 of 32 districts in NYC were in the bottom 10% in the state in Science; 26 of 32 districts were at the bottom 10% in the state in Social Studies.

I’m sure there’s more deception and spin in these slides; please take a look and leave a comment on the comment page!

NY Times falls in line with the Bloomberg PR spin control

This New York Times article on the Bloomberg/Klein record on test scores is incomplete, biased, and in some cases, clearly inaccurate.

1- The article falsely claims that fourth-grade reading scores rose on the national exams called the NAEPs, whereas there has been no significant gain since 2003, when the “Children First” reforms were introduced.

2- As reported in our book, “NYC Schools under Bloomberg and Klein: What Parents, Teachers and Policymakers Need to Know,” there has been no closing of the achievement gap between racial or ethnic groups in any grade or subject tested by the NAEPs, considered by experts the "gold standard" of assessments.

3- The article fails to mention that in fourth grade math, the one area that did show a significant increase in the NAEPs, the student exclusion rate was 25 percent, far higher than any other city tested.

4- The article fails to mention that in 8^th grade reading, NYC made less progress on the NAEPs than any of the nine other cities tested.

5- The article fails to quote any independent experts or point to any evidence of test score inflation on the state exams, including lower cut scores, the repetition of similar questions, or the narrowing of topics covered, as revealed in articles in the Daily News and elsewhere.

Instead, the article quotes at great length their favorite expert, Howard Everson, head of a panel assembled by the State Education Department, who says that “New York’s tests were “about as good as we can build them,” as well as Merryl Tisch, the head of the Regents, and Joel Klein himself, all interested parties who predictably claim that the results indicate actual improvements in learning.

And though the system is full of teachers and even principals who will attest to the fact that the exams have become easier over time, the reporter chooses to quote not a single one.

Given the evident bias of this article, it appears clear that the Times has been captured by the Bloomberg/Klein PR machine, and can no longer be trusted to provide objective analysis of their education record.

If you’d like to chime in, you could ask for a correction at nytnews@nytimes.com; write a letter to the editor at letters@nytimes.com, and/or email the public editor at public@nytimes.com