rheeForm

Proposed education reforms that do not imagine that current and beginning teachers can become more effective while on the job should be considered null and void. This postulation, if accepted, would direct Michelle Rhee's new StudentsFirst agenda to the nearest paper shredder.

To be blunt, it is just plain naive and short-sighted to think that we can maximize teacher effectiveness purely by firing more teachers and marginally changing the cadre of incoming teacher candidates. Is supporting and strengthening the teaching practice of our veteran educators not worthy of our focus and investment?

StudentsFirst's "Elevate Teaching" policy objectives are limited to evaluating teachers and principals, reforming teacher certification laws, reforming teacher compensation, "exiting" teachers, and eliminating teacher tenure. Specifically, the objectives are:

• State law must require evaluation that is based substantially on student achievement. Evaluation tools should measure at least half of a teacher's performance based on student achievement, using a value-added growth model. The other aspects of a teacher's evaluations should derive from measures that align with student results, including high-quality observations and student evaluations of teacher practice.
• To avoid all teachers being ranked as effective without meaningful assessment, evaluations must anchor effectiveness around a year's worth of growth.
• State law must require principal evaluation that is based on student achievement and effective management of teachers. Districts should evaluate at least half of a school administrator's performance based on student achievement, and the remaining portion should mostly relate to their ability to attract, retain, manage, and develop excellent teachers.
• State law should give districts the autonomy to develop teacher evaluation systems apart from the collective bargaining process. Evaluations should be a matter of district policy.
• States must reduce legal barriers to entry in the teaching profession, including complicated credentialing or certification schemes that rely upon factors that do not clearly correlate with teacher effectiveness.
• State law should not be structured to penalize districts financially for recruiting teachers from alternate certification programs.
• States should adopt a clear process by which alternative certification programs are authorized, continually evaluated, and decommissioned if not producing high-quality educators.
• State law must facilitate digital learning by allowing certification for online instruction and modifying or eliminating mandatory "seat time" laws.
• State law must require pay structures based primarily on effectiveness. Teacher contracts must allow for individual performance-based pay.
• State law and district policy should not mandate higher salaries for master's degrees or additional education credits.
• State law should require staffing decisions (transfers, reductions, placements) be based on teacher effectiveness.
• State laws must prohibit forced placements and allow district control in staffing. Districts should ensure that teacher contracts require mutual consent placements. Districts should have the flexibility to offer defined grace periods, severance, or other options for teachers who have effective ratings, but do not find a mutually agreeable placement. Teachers rated ineffective should be exited from the system.
• State law should not grant, implicitly or directly, tenure or permanent contracts for PK–12 education professionals.

There is evidence (from sources such as IES and AIR) that shows that high-quality approaches to new teacher induction and professional development pay dividends in terms of student outcomes. Why would a "student first" agenda utterly ignore initiatives that work in favor of some that have a paltry or non-existent research base?

To keep it brief, please read some of my most relevant past posts arguing why a focus on teacher support and development makes sense and why it should be at the centerpiece of every education reform agenda.

• Teacher Quality: What You Need To Know
• Adding Value To The Value-Added Debate
• Positive Effects Of Comprehensive Teacher Induction
• State Teacher Policies Suck!
• Race To The Top: Under The Hood
• Research: Attracting New Teachers To Urban Schools
• Thoughts On Equitable Teacher Distribution
• Measurement Is Not Destiny
• Does Mentoring Improve New Teachers?
  

With regard to the StudentsFirst plan, to use a Twitter construct, #edreformfail.

A New Year

We hope all our readers enjoyed relaxing holidays and have returned refreshed for the new year. While our family and professional lives continue to make it difficult to blog with great frequency, we hope you'll continue to read our infrequent commentary and join in the discussion during 2011.

A few thoughts to start the new year...

(1) Outcomes First? If outcomes are what really matter in education, it is interesting that so many advocates, commentators and policy organizations seem to count adoption of favored policy reforms as ends in themselves. We are all guilty of this to some degree. It is only when there is a research base to suggest that specific reforms and programs work that there is a strong argument to be made. Examples might include targeted class size reduction in grades k-3, high-quality early childhood education, and comprehensive, multi-year induction support for new teachers. But, at a macro level, certain arguments fall apart when there is no evidence to back them up, such as teachers' unions being a wart on the ass of progress. Take Massachusetts, for example, a strong union state. It leads the nation in TIMMS scores in spite of the fact that the Massachusetts Teachers Association looms large in state politics.

(2) Teachers, Teachers: One of the best developments of 2010 was an increased focus on teachers and on teacher effectiveness in particular. This focus was not always for the better, as in the case of the Los Angeles Times' decision to publish value-added scores for individual teachers or the misleading, union-bashing documentary Waiting For Superman. But an overall focus on the outcomes of teaching is the right policy conversation to be having. However, that conversation must lead to solutions that create comprehensive structures and systems to maximize benefits for all involved -- students, teachers, parents, etc. Regular feedback about teaching is critical for educators, not just summative data or annual evaluations that don't provide actionable feedback. A key goal around improving teacher effectiveness should be the development of schools and districts as communities of practice that make teaching more of a collective endeavor and support all educators to strengthen their individual practices and skills.

(3) ESEA: I am increasingly of the mind that something -- but not much of anything -- will happen with regard to reauthorization of the Elementary and Secondary Education Act in 2011. If successful, reauthorization will primarily serve as a token of bipartisanship that both parties can carry into the 2012 elections to say "we can work together to get things done." If accomplished, it may be one of the few significant bipartisan accomplishments of this Congress. Look for substantive tweaks to the No Child Left Behind Act rather than a wholesale overhaul of it. More flexibility around AYP. Attention to the needs of rural districts. More local control. Perhaps a stronger focus on teacher performance pay, charter schools and school choice options -- some elements of the Obama Blueprint combined with priority issues for Republicans like John Kline and Lamar Alexander. And level funding, at best.

(4) Exclusivity: One of my wishes for the New Year is that the DC echo chamber would become less and less influential in conversations about education policy. I am constantly amazed at how regularly the usual suspects parrot, squawk about and retweet the comments and ideas of the other usual suspects, especially those with whom they have personal or proprietary relationships. And how the same usual suspects are quoted saying the same usual things by the mainstream and educational media. This dynamic plays out, too, in conversations within multiple exclusive fiefdoms within education that generally have little to no intersection with fiefdoms with competing worldviews or different policy priorities. As someone who once worked in DC and who now works for a non-DC-based national non-profit organization that has relationships with all sides of the education community, I am especially cognizant of this dynamic in which voices outside of the Beltway 'influentials' are not heard.

One alternative stream recently profiled by Rick Hess and Jay Greene are academics doing policy-relevant research and cutting a high profile in policy conversations. We need more of that type of intellect in play -- and not just from economists. Another is the rise of state-based reform groups like Stand for Children, the PIE Network, Delaware's Rodel Foundation and Oregon's Chalkboard Project. Finally, the voice of actual teachers is too often missing from policy conversations. Fortunately, there are numerous efforts afoot to remedy this. Two, in particular, worth checking out are Teach PLUS and the VIVA Project. My organization, the New Teacher Center, in conjunction with the College Board, recently profiled real-life teachers in a publication about the importance of teacher mentoring.

One way or another, 2011 undoubtedly will be an interesting year for education.

Building A Better Teacher

If you haven't been reading the excellent "Building A Better Teacher" news series in the Milwaukee Journal-Sentinel, you should be. It really doesn't matter whether you're from Wisconsin or not, or particularly interested in this state's policy context. The series is taking an expansive look at the various issues related to human capital development, teacher effectiveness and teaching quality. And it's not quoting the same overused Beltway prognosticators to drive its points home.

The fourth installment in the eight-part series, funded by Hechinger, ran this past Sunday and was entitled "Trying to steer strong teachers to weak schools."

My main quibble with this particular article was that it gave short shrift to one of the most effective answers to the question posed: How do we steer strong teachers to weak schools? The answer: Improve the teaching conditions at those schools.

Here's the extent of what the article offered on this issue:

So what else might be done, in hopes of having more impact? A few ideas in nutshells:

• Make schools better places to work: This is both the simplest and most complex solution. The New Teacher Project report in 2007 said, "The best way to staff high need schools is to make them attractive to great teachers." But how do you achieve that?

Mike Langyel, president of the Milwaukee teachers union, listed things that would attract teachers: "A competent and fair principal is key not only in getting teachers there but in keeping them.... We're also looking at schools that are safe."

My suggestion would have been a much more robust treatment and discussion of the issue of teaching conditions. I have extrapolated on its importance in a series of blog posts, and the New Teacher Center (my employer) has unique national expertise in administering statewide Teaching and Learning Conditions surveys. The NTC has a grant from the Bill & Melinda Gates Foundation to administer a Teaching & Learning Conditions Survey as part of the foundation’s Measures of Effective Teaching (MET) project. The Survey is being administered in select schools and districts participating in the MET project across the country.

Perhaps Wisconsin and Milwaukee, in particular, should consider administering such an anonymous full population survey to its educators -- teachers, administrators and support staff -- and see what they have to say. Why do they stay or leave a given school or district? What's working and what isn't? States and districts that have administered such surveys have used the data to improve principal preparation, rewrite professional standards for teachers and principals, and strengthen teacher mentoring and professional development. This is not data to be afraid of but data that can empower policymakers, school leaders and teachers alike.

Teaching and learning conditions are highly correlated with issues such as teacher retention and the presence of such conditions explain as much as 15 percent of the variance in student achievement between schools (Helen 'Sunny' Ladd, 2009). This stuff matters greatly in the current policy debates about teaching and student outcomes and it gets far too little attention as compared with value added, teacher evaluation and teacher pay.

Misleading Manifesto

I'm sorry, but the "manifesto" published in today's Washington Post really pisses me off because it is built upon a false premise. It is authored by a number of urban school superintendents, including Chicago's Ron Huberman, New York City's Joel Klein, Washington DC's Michelle Rhee, and New Orleans' Paul Vallas. And it -- intentionally? -- misstates educational research.

"[T]he single most important factor determining whether students succeed in school is not the color of their skin or their ZIP code or even their parents' income -- it is the quality of their teacher."

No. That is patently false.

Now, listen here. I work for a teacher-focused, non-profit organization, the New Teacher Center (NTC). Wouldn't it be powerful to go out and say that teachers matter more than ANYTHING else? But they don't. In terms of school-based variables, they do. But in terms of all variables that impact students, they simply do not. No research says that. In our messaging at the NTC, we are always careful to say that teacher quality is the most important school-based variable for student achievement (examples here and here (on page 4)). That's accurate, honest and powerful in its own right.

So why not make the case for improving teaching in a honest fashion? There is an incredibly strong case to make that improving teaching quality is a critically important and policy amenable part of the solution to increasing student achievement and closing achievement gaps for disadvantaged students. But it's only part of the answer which requires solutions beyond the educational system. Let's not lose sight of that.

At the Shanker Blog, Matthew Di Carlo explored this same issue last month and took journalists to task for making similar claims. Back in July, he summarized existing teacher quality research.

September 16, 2010:

The same body of evidence that shows that teachers are the most important within-school factor influencing test score gains also demonstrates that non-school factors matter a great deal more. [emphasis added] The first wave of high-profile articles in our newly-energized education debate not only seem to be failing to provide this context, but are ignoring it completely. Deliberately or not, they are publishing incorrect information dressed up as empirical fact, spreading it throughout a mass audience new to the topic, to the detriment of us all.

Even though the 10-15 percent explained by teachers still represents a great deal of power (and is among the only factors “within the jurisdiction” of education policy), it is nevertheless important to bear in mind that poor educational outcomes are a result of a complicated web of social and economic forces. [emphasis added] People have to understand that, or they will maintain unrealistic expectations about the extent to which teacher-related policies alone can solve our problems, and how quickly they will work.

July 14, 2010:

But in the big picture, roughly 60 percent of achievement outcomes is explained by student and family background characteristics (most are unobserved, but likely pertain to income/poverty). [emphasis added] Observable and unobservable schooling factors explain roughly 20 percent, most of this (10-15 percent) being teacher effects. The rest of the variation (about 20 percent) is unexplained (error). In other words, though precise estimates vary, the preponderance of evidence shows that achievement differences between students are overwhelmingly attributable to factors outside of schools and classrooms (see Hanushek et al. 1998; Rockoff 2003; Goldhaber et al. 1999; Rowan et al. 2002; Nye et al. 2004).

Let's take Di Carlo's and Joe Friday's advice. Just the facts, please.

LA Times Value Added Editorial

The Los Angeles Times editorial page gets it mostly right today on the value-added issue ("Good teachers, good students," September 3, 2010). It says a number of smart things that I agree with, such as:

• "Test scores are indeed just one indicator of a teacher's performance."
• "But it's revealing, and disturbing, to read the comments of some teachers who don't seem to care whether their students' scores slide. They argue that they're focused on more important things than the tests measure. That's unpersuasive."
• "This page has never believed that test scores should count for all of a teacher's evaluation — or even be the most important factor. But they should be a part of it."
• "Right now, the "value-added" scores The Times has been reporting are more useful for evaluating schools than teachers. Many factors can throw off the data at the classroom level."
• "That's why we think the Obama administration has been too hasty to push states into linking test scores to teacher evaluations and to reward states that overemphasize the scores, making them count for half or more of a teacher's worth. The administration's first priorities should have been developing better tests, which it's working on now — if we're going to judge teachers in part by these scores, it's unacceptable to say that top-notch tests are too expensive — and statistical models that minimize random factors and make the scores a better evaluation tool."
• "Current teacher evaluation practices are ripe for overhaul. Performance reviews should include, at minimum, classroom observations, portfolios of student work over the academic year and, yes, objective test data."

I just wish its news division had taken some of these points to heart, namely having patience until the methodology was ready to be joined by other measures of teacher effectiveness, such as classroom observations, and not publishing the value-added scores of individual teachers and definitively labeling some as most effective and least effective.

Heather Horn of The Atlantic magazine offers a nice summary of some of the related issues and links to relevant sources in this September 1, 2010 blog post.

And Dana Goldstein offers a smart retort (and a preview of her upcoming The Nation feature on value added?) to a vacuous and vitriolic Slate post by Jack Shafer on this topic.

Related Posts:

• Education Optimists: "Adding Value to the Value-Added Debate" (August 31, 2010)

• Education Optimists: "More Grist for the Value-Added Mill" (September 1, 2010)

• Education Optimists: All "value added" related posts

More Grist for the Value-Added Mill

Here is additional smart and pithy commentary on the current value-added conversation that I wasn't able to incorporate into yesterday's post or have only discovered since.

• Dr. Douglas Harris - "Not By Value Added Alone" (The Blog of Harvard Education Publishing)
• Dr. Aaron Pallas - "Value-added measures: The cardinal rule and the cardinal sin" (A Sociological Eye on Education)
  
• Stephen Sawchuk - "Some Scholars Slam Value-Added for Teacher Accountability" (Teacher Beat/Education Week)
• Jay Mathews - "America's Best Teacher and the L.A. Times" (Class Struggle/Washington Post)
• Dr. Daniel Willingham - "3 Key Factors in Teacher Evaluation" (The Answer Sheet/Washington Post)
• Sherman Dorn - "Please leave your magic numbers on the magic carpet with your magic wand" (Sherman Dorn)
• Bill Tucker - "My Value-Added Number" (Quick and the Ed)
• John Thompson - "High-Ranking Teachers Ignore Pacing Standards" (This Week In Education)
• Walt Gardner - "Why Not Name and Shame Teachers?" (Walt Gardner's Reality Check/Education Week)
• Claus von Zastrow - "The LA Times Goes Astray" (Public School Insights/Learning First Alliance)
• Alan Borsuk - "Where newspaper goes in rating teachers, others soon will follow" (Milwaukee Journal Sentinel)
• Andy Rotherham - "LA Story" (Eduwonk)

Adding Value to the Value-Added Debate

Seeing as I am not paid to blog as part of my daily job, it's basically impossible for me to be even close to first out of the box on the issues of the day. Add to that being a parent of two small children (my most important job – right up there with being a husband) and that only adds to my sometimes frustration of not being able to weigh in on some of these issues quickly.

That said, here is my attempt to distill some key points and share my opinions -- add value, if you will -- to the debate that is raging as a result of the Los Angeles Times's decision to publish the value-added scores of individual teachers in the L.A. Unified School District.

First of all, let me address the issue at hand. I believe that the LA Times's decision to publish the value-added scores was irresponsible. Given what we know about the unreliability and variability in such scores and the likelihood that consumers of said scores will use them at face value without fully understanding all of the caveats, this was a dish that should have been sent back to the kitchen.

Although the LA Times is not a government or public entity, it does operate in the public sphere. And it has a responsibility as such an actor. Its decision to label LA teachers as 'effective' and 'ineffective' based on suspect value-added data alone is akin to an auditor secretly investigating a firm or agency without an engagement letter and publishing findings that may or may not hold water.

Frankly, I don't care what positive benefits this decision by the LA Times might have engendered. Yes, the district and the teachers union have agreed to begin negotiations on a new evaluation system. Top district officials have said they want at least 30% of a teacher's review to be based on value-added and have wisely said that the majority of the evaluations should depend on classroom observations. Such a development exonerates the LA Times, as some have argued. In my mind, any such benefits are purloined and come at the expense of sticking it -- rightly in some cases, certainly wrongly in others -- to individual teachers who mostly are trying their best.

Oh, I know, I know. It's not about the teachers anymore. Their day has come and gone. "It's about the kids" now, right? But you know what? The decisions we make about how we license, compensate, evaluate and dismiss teachers affects them as individual people, as husbands and wives, as mothers and fathers. It effects who may or may not choose to enter the profession in the coming years. If we mistakenly catch a bunch of teachers in a wrong-headed, value-added dragnet based upon a missionary zeal and 'head in the sand' conviction that numbers don't lie, we will be doing a disservice both to teachers and to the kids. And, if we start slicing and dicing teachers left and right, who exactly will replace them?

(1) Value-added test scores should not be used as the primary means of informing high-stakes decisions, such as tenure and dismissal.
One primary piece of evidence was released just this week from the well-respected, nonpartisan Economic Policy Institute. The EPI report, co-authored by numerous academic experts, said:

• Student test scores are not reliable indicators of teacher effectiveness, even with the addition of value-added modeling (VAM).
  
• Though VAM methods have allowed for more sophisticated comparisons of teachers than were possible in the past, they are still inaccurate, so test scores should not dominate the information used by school officials in making high-stakes decisions about the evaluation, discipline and compensation of teachers.
  
• Neither parents nor anyone else should believe that the Los Angeles Times analysis actually identifies which teachers are effective or ineffective in teaching children because the methods are incapable of doing so fairly and accurately.
• Analyses of VAM results show that they are often unstable across time, classes and tests; thus, test scores, even with the addition of VAM, are not accurate indicators of teacher effectiveness. Student test scores, even with VAM, cannot fully account for the wide range of factors that influence student learning, particularly the backgrounds of students, school supports and the effects of summer learning loss. As a result, teachers who teach students with the greatest educational needs appear to be less effective than they are.

Other experts, such as Mathematica Policy Research, Rick Hess, and Dan Goldhaber have offered important cautions as well.

The findings of the IES-funded Mathematica report were “largely driven by findings from the literature and new analyses that more than 90 percent of the variation in student gain scores is due to the variation in student-level factors that are not under the control of the teacher. Thus, multiple years of performance data are required to reliably detect a teacher’s true long-run performance signal from the student-level noise…. Type I and II error rates [‘false positives’ and ‘false negatives’] for teacher-level analyses will be about 26 percent if three years of data are used for estimation. In a typical performance measurement system, more than 1 in 4 teachers who are truly average in performance will be erroneously identified for special treatment, and more than 1 in 4 teachers who differ from average performance by 3 months of student learning in math or 4 more in reading will be overlooked. In addition, Type I and II error rates will likely decrease by only about one half (from 26 to 12 percent) using 10 years of data.”

Hess has “three serious problems with what the LAT did. First … I'm increasingly nervous at how casually reading and math value-added calculations are being treated as de facto determinants of "good" teaching…. Second, beyond these kinds of technical considerations, there are structural problems. For instance, in those cases where students receive substantial pull-out instruction or work with a designated reading instructor, LAT-style value-added calculations are going to conflate the impact of the teacher and this other instruction…. Third, there's a profound failure to recognize the difference between responsible management and public transparency.”

Goldhaber, in a Seattle Times op-ed, says that he “support[s] the idea of using value-added methods as one means of judging teacher performance, but strongly oppose[s] making the performance estimates of individual teachers public in this way. First, there are reasons to be concerned that individual value-added estimates may be misleading indicators of true teacher performance. Second, performance estimates that look different from one another on paper may not truly be distinct in a statistically significant sense. Finally, and perhaps most important, I cannot think of a profession in either the public or private sector where individual employee performance estimates are made public in a newspaper.”

Multiple measures to inform teacher evaluation seems like the right approach, including the use of multiple years of value-added student data (one thing the LA Times DID get right). That said, the available research would seem to suggest that states (particularly in Race to the Top) that have proposed basing 50% or more of an individual educators evaluation on a value-added score may have gone too far down the path. LA Unified officials have said (LA Times, 8/30/2010) they want at least 30% of a teacher's review to be based on value-added and that the majority of the evaluations should depend on observations. That might be a more appropriate stance.

(2) Embracing the status quo is unacceptable.
As reports such at The New Teacher Project's Widget Effect have chronicled, current approaches to teacher evaluation are broken. They don’t work for anyone involved. Critics of VAM cannot simply draw a line in the sand and state that, "This will not stand!" If not this, then what? Certainly not the current system! Fortunately, efforts led by organizations such as the American Federation of Teachers and the Hope Street Group are developing or have offered thoughtful solutions to this issue. [Disclosure: I participated in Hope Street's effort and my New Teacher Center colleague Eric Hirsch serve on AFT’s evaluation committee.] Sadly, LA Unified and the LA Teachers Union both are culpable –along with the LA Times – in bringing this upon the city's teachers by refusing to act to analyze or utilize available value-added data. An adherence to the status quo created a void that the LA Times sought to fill in order to sell more newspapers in a wrong-headed attempt to inform the public.

(3) The ‘lesser of two evils’ axiom should not be invoked.
Even if you agree that all the factors we currently use to select and sort teachers is worse than a value added only alternative, as argued by Education Sector's Chad Aldeman, our current arsenal does not meaningfully inform high-stakes decisions (apart from entry tests with largely low passing scores and the aforementioned impossible-to-fail evaluations). That's, of course, both a condemnation of the current system's inability and/or unwillingness to differentiate between teachers, but it's also a recognition that we haven't struck the right balance or developed the value-added systems to inform high-stakes decisions in this regard in all but a few promising places.

(4) Don't lose sight of the utility of value-added data to inform formative assessment of teaching practice.
If one of the takeaways from research is that value-added data shouldn't be used to drive high-stakes decisions, it is helpful to think about the use of this data to inform teacher development. Analysis of student work, including relevant test scores, is an important professional development opportunity that all teachers, especially new ones, should have regular opportunities to engage in. Systems such as the NTC’s Formative Assessment System provide such a tool in states and districts with whom it works on teacher induction. Sadly, this is not the norm in American schools, but is built into high-quality professional development approaches, as Sara Mead wisely discusses in her recent Ed Week blog post. As I noted under #2, LA Unified missed an opportunity to embrace such data to inform its educators in such a way. In the LA Times value added series, several teachers bemoaned the fact that they had never had the opportunity to see such data until it was published in the newspaper.

(5) Valid and reliable classroom observation conducted by trained evaluators is critical.
Other elements of an evaluation system are even more important than value-added methodology if for no other reason that the majority of teachers do not teach tested subjects. Unless we, God forbid, develop multiple-choice assessments of more and more subjects and grade levels, we're going to need valid and reliable ways of assessing the practice of educators who cannot be assessed by value-added student achievement scores. Despite some of the criticisms lobbed at the District of Columbia's new IMPACT evaluation system, this is an element at the heart of DC’s approach to teacher evaluation. Further, the Gates Foundation’s on-going teacher effectiveness study holds great promise.

(6) We've got to get beyond this focus on the 'best' and 'worst' teachers.
How about we focus on strengthening the effectiveness of the 80-90% of teachers in the middle? We know how to do that through comprehensive new teacher induction and high-quality professional development, but we're just lacking the collective will to pull it off and invest in what makes a difference. These are similar roadblocks to what has prevented the use of student outcomes from being considered in teacher evaluations. It raises discomfort, requires a change in prevailing (often mediocre) practices, demands greater accountability, and necessitates viewing teaching not as a private activity but as a collective endeavor. But I keep making this point over and over again about the importance of a teacher development focus within the teacher effectiveness conversation because I see too few reform advocates taking it seriously. Take off the blinders, folks. It is not primarily about firing teachers.

(7) Teacher effectiveness is contextual.
Teaching and learning conditions impact an individual educator’s ability to succeed. It is entirely possible that an individual teacher's value-added score is significantly determined by the teaching and learning conditions (supportive leadership, opportunities to collaborate, classroom resources) present at their school site than about their individual knowledge, skills and practices. In Seinfeldian terms, teachers are not 'masters of their domain' necessarily. The EPI report makes this point. So do my New Teacher Center colleagues through statewide teaching and learning conditions surveys. So does Duke University economist Helen Ladd (also a co-signed on the EPI report) and the University of Toronto’s Kenneth Leithwood.

Positive Effects of Comprehensive Teacher Induction

Today, Mathematica Policy Research, Inc. released the final report of its IES/U.S Department of Education-funded randomized controlled trial (RCT) of comprehensive teacher induction. It shows a statistically significant and sizeable impact on student achievement in mathematics (0.20 standard deviations) and reading (0.11 standard deviations) of third-year teachers who received two years of robust induction support. That's the equivalent of moving students from the 50th to 54th percentile in reading achievement and from the 50th to 58th percentile in math achievement.

As a basis of comparison, I note that in 2004, Mathematica conducted a RCT of Teach for America (TFA). In that study, it compared the gains in reading and math achievement made by students randomly assigned to TFA teachers or other teachers in the same school. The results showed that, on average, students with TFA teachers raised their mathematics test scores by 0.15 standard deviations (versus 0.20 standard deviations in the induction study), but found no impact on reading test scores (versus 0.11 standard deviations in the induction study).

In another recent Mathematica report (boy, these folks are busy!), the authors note that "The achievement effects of class-size reduction are often used as a benchmark for other educational interventions. After three years of treatment (grades K-2) in classes one-third smaller than typical, average student gains amounted to 0.20 standard deviations in math and 0.23 standard deviations in reading (U.S. Department of Education, 1998)." In that report -- an evaluation of the Knowledge Is Power Program (KIPP), Mathematica researchers found a very powerful impact from KIPP: "For the vast majority of KIPP schools studied, impacts on students’ state assessment scores in mathematics and reading are positive, statistically significant, and educationally substantial.... By year three, half of the KIPP schools in our sample are producing math impacts of 0.48 standard deviations or more, equivalent to the effect of moving a student from the 30th percentile to the 48th percentile on a typical test distribution..... Half of the KIPP schools in our sample show three-year reading effects of 0.28 standard deviations or more."

Is it appropriate to compare effect sizes among RCTs or, for that matter, among research in general? I am told that it is, although certainly considerations such as cost effectiveness and scalability have to enter into the conversation. Implementation issues also must be attended to. With regard to teacher induction, the issue of cost effectiveness was addressed in a 2007 cost-benefit study published in the Education Research Service's Spectrum journal and summarized in this New Teacher Center (NTC) policy brief.

Disclosure: I am employed by the NTC which participated in the induction RCT, and I helped to coordinate NTC's statement on the study.

The NTC is "encouraged" by the study. However, NTC believes that "it does not reflect the even more significant outcomes that can be achieved when districts have the time, capacity and willingness to focus on an in-depth, universal implementation of comprehensive, high-quality induction. It speaks volumes about the quality of induction and mentoring provided and the necessity of new teacher support that student achievement gains were documented despite [design and implementation] limitations to the study."

UPDATE: Read the Education Week story by Stephen Sawchuk here. And the Mathematica press release here.

Where to Begin?

Where do I begin in critiquing such wrong-headed and vitriolic analysis of Florida Governor Charlie Crist's veto of SB 6, which would have eliminated tenure for teachers and based their evaluations primarily on a single year of student test scores?

One cannot accurately and fairly evaluate an individual educator's performance by test scores alone, especially based on a single year's worth of data (as the Florida legislation would have done) and particularly for new teachers. I've said it before and before that -- and I'll undoubtedly say it again. On this specific issue, I'll take the "what they said" approach. Read Claus von Zastrow, Sherman Dorn, David Kirp, and Steve Peha who provide the right amount of counsel and insight. Today's blog post by Rick Hess on value-added methodologies is also worth reading.

But, first, I'll say a little more. It appears that a fair number of the forces pushing SB 6 and now bemoaning its veto admit that it was a flawed bill. But, those parties then say, ANYTHING is better than the current system. No doubt the current system needs fixin', but why didn't Florida policymakers craft legislation that took a more nuanced view of teacher effectiveness and which recognized the shortcoming of evaluating teachers based on one year's worth of test scores? Dear Florida and Dear Reformers: Just because you believe the status quo sucks doesn't excuse your lack of effort in designing a reform that sucks just slightly less.

As for politics, it seems like, if anything, especially as Crist is a candidate for US Senate in a Republican primary, this veto was an act of political courage. How former Bushie John Bailey can claim it represents the opposite seems to me to represent fanciful, Republican-style logic that might play at a Sarah Palin rally, but that this Education Optimist ain't buying.

And, as for claims (here and here), that Crist's veto will cost Florida funding in Phase Two of the Race to the Top competition, check back here in September. I predict that Florida will be holding a bag of money at that point, this veto notwithstanding.

ESEA Hearing on Teachers and Leaders

The U.S. Senate Committee on Health, Education, Labor and Pensions is holding a hearing this morning on the reauthorization of the Elementary and Secondary Education Act (ESEA), currently known as No Child Left Behind. Today's hearing focuses on the teacher and school leader elements of ESEA.

Among the witnesses are:

• Randi Weingarten, President, American Federation of Teachers
• Stephanie Hirsch, Executive Director, National Staff Development Council
  
• Jon Schnur, CEO, New Leaders for New Schools
  
• Ellen Moir, CEO, New Teacher Center
  
• Timothy Daly, President, The New Teacher Project
• Thomas Kane, Professor of Education and Economics, Harvard University and Deputy Director, U.S. Education, Bill & Melinda Gates Foundation
  

UPDATES:
A video replay of the hearing -- as well as links to the participants' testimony -- is available here.

Teaching and Learning Conditions

I'm catching up on education news and blogging after some well-spent time with our family in New York and Vermont last week....

Both successful Phase One Race to the Top (RttT) states -- Delaware and Tennessee -- plan to conduct a statewide teacher working conditions survey. Was this the secret to each state's victory? Well, not exactly, as the states of Colorado, Kentucky, Massachusetts, North Carolina and Ohio also built such a survey into their applications. Of course, each of those states were among the 16 Phase One semifinalists. So, maybe there is something there.

Independent of RttT, however, such efforts are in line with President Obama’s recent Blueprint for Reform: The Reauthorization of the Elementary and Secondary Education Act, which would require states and districts to collect and report teacher survey data on available professional support and working conditions in schools biennially.

Research has demonstrated a connection between positive teaching and learning conditions, teacher retention, and student achievement.

• “There is good evidence to show that teachers’ working conditions matter because they have a direct effect on teachers’ thoughts and feelings—their sense of individual professional efficacy, of collective professional efficacy, of job satisfaction; their organizational commitment, levels of stress and burnout, morale, engagement in the school or profession and their pedagogical content knowledge. These internal states are an important factor in what teachers do and have a direct effect in what happens in the classroom, how well students achieve, and their experience of school.” (Leithwood, 2006)
  
• “Working conditions emerge as highly predictive of teachers’ stated intentions to remain or leave their schools, with leadership emerging as the most salient dimension. Teachers’ perceptions of their working conditions are also predictive of one-year actual departure rates and student achievement, but the predictive power is far lower…Taken together, the working conditions variables account for 10 to 15 percent of the explained variation in math and reading scores across schools, after controlling for individual and school level characteristics of schools.” (Ladd, 2009)
• “[O]ur analysis of teacher mobility showed that salary affects mobility patterns less than do working conditions such as facilities, safety and quality of leadership.” (Hanushek and Rivkin, 2007)
• “…working conditions factors, especially principal support, had more influence on simulated job choice than pay level, implying that money might be better spent to attract, retain or train better principals than to provide higher beginning salaries to teachers in schools with high-poverty or a high proportion of students of color.” (Milanowski et al., 2009)
• A survey of 2,000 educators from California found that 28 percent of teachers who left the profession before retirement indicated that they would come back if improvements were made to teaching and learning conditions. (Futernick, 2007)

Last week's press release from the New Teacher Center goes into greater detail:

“Research has shown that understanding and improving teaching and learning conditions results in increased student success, improved teacher efficacy and motivation, higher teacher retention, and better recruitment strategies that bring educators to hard-to-staff schools,” said Ellen Moir, Chief Executive Officer of the New Teacher Center. “In the past, policymakers have not had the data necessary they need to address educators’ working conditions. Our surveys change this by putting valuable information in the hands of people who make important decisions every day that impact our schools and all those who work and learn in them.”

The New Teacher Center (NTC) assists states and school districts in administering the anonymous, web-based Teaching and Learning Conditions Survey. The NTC has a proven track record of successful administration of teaching and learning conditions surveys in 15 states. In addition to working with state stakeholders to design a customized survey, NTC provides analyses and training materials to help all stakeholders understand and use the Teaching and Learning Conditions Survey results for school improvement.

The Teaching & Learning Conditions Survey has the longest history in North Carolina where policymakers at different levels have utilized Survey data in different ways. Local education leaders have used results at the district level to further bond initiatives. At the state level, data was used in rewriting standards for principals and teachers. The Survey initiative has been so expansive that it has supported the creation of additional funding for professional development in low-performing schools. Results also have led to the development of school leadership training which requires administrators to use Survey data in making school-level improvement decisions.

The news article ('Teacher Surveys Aimed at Swaying Policymakers') from Education Week's Stephen Sawchuk provides additional context:

Despite their differing sample sizes and specific questions, the surveys’ findings about what teachers say they need to be successful are remarkably consistent from instrument to instrument. Some of the top findings: Teachers report that the quality of their schools’ leadership, a say in school decisionmaking, and opportunities to work with their peers affect their own capacity as educators.

You're Fired!

I am deeply troubled to read columns like this ("Improve education, fire bad teachers") -- both the title and the content -- from a reputable source like the Center for American Progress (CAP). Much as the likes of FOX News are in desperate need of balance and breadth of perspective, so is this column.

Where is the discussion about the need to support teachers to become more effective through improved preparation, stronger induction and mentoring, and job-embedded professional development? What about more than a throwaway line about the role of teacher evaluation systems to provide constructive feedback to help teachers identify strengths and weaknesses and help them become more effective?

I don't mean to pick on CAP too harshly, for some of its prior reports (such as this one) approached the teacher effectiveness issue more comprehensively and accurately. But if all we do is focus on firing teachers, without addressing other elements of teacher quality policy, we're going to dig ourselves into a hole that we'll never crawl out of. While stricter license and tenure requirements and more meaningful teacher evaluation systems might weed out truly ineffective teachers (a small minority), it won't do anything to help the vast majority teachers become more successful without a clear focus on individualized teacher development.

Another recent example of oversimplification and the repetitive 'teachers suck' mantra appeared on the pages of Newsweek masquerading as an actual news article. (I'm glad I canceled my subscription years ago.) The authors pontificated that, "Nothing, then, is more important than hiring good teachers and firing bad ones." Um, OK. Nothing, huh?

We need a broader vision here, folks, along the lines that the Obama Administration has articulated in its initial ESEA blueprint. It is not as simple as just firing more teachers. Columns like these do not convey the complexity and comprehensiveness of the policies, practices and implementation that is needed to truly improve teacher effectiveness across the board. They simplify the problem and cast the responsibility for educational failure solely on teachers.

Speaking of balance, here are more of my thoughts....

UPDATE: Eduwonk and Claus von Zastrow make good points on this issue -- as does Diane Ravitch (here and here). Bill Maher offers his own 'new rule, raising the important issue of parental involvement.

Live from San Jose

Tomorrow I'll be Twittering from the New Teacher Center's 12th National Symposium in San Jose, California.

Tuesday's agenda features a morning keynote panel on measuring teacher effectiveness. Panelists include Terry Holliday, Kentucky Commissioner of Education; Brad Jupp, Senior Program Advisor, Teacher Effectiveness and Quality, U.S. Department of Education; and Tom Kane, Professor of Education & Economics, Harvard University and Deputy Director for Education, Gates Foundation. It is moderated by my intrepid NTC colleague, Eric Hirsch.

Today, keynoters include Linda Darling-Hammond and Richard Rothstein, but my schedule likely won't allow me to have my thumbs affixed to my phone or laptop during their presentations.

To follow me, here is the link to my Twitter account.

State Teacher Policies Suck!

I'm sure glad that Kate Walsh and company weren't my professors in college. Damn! They are tough graders! With the exception of eight southern states (Alabama, Arkansas, Florida, Georgia, Louisiana, South Carolina, Tennessee, Texas) that received a 'C' and three northern states (Maine, Montana, Vermont) that received a 'F', every U.S. state received some version of a 'D' in the latest edition of the National Council on Teacher Quality's State Teacher Policy Yearbook. In grading the states, the authors look at five broad teacher quality areas (and numerous metrics within them): teacher preparation, expanding the pool of teachers, identifying effective teachers, retaining effective teachers, and exiting ineffective teachers.

While it is easy to poke holes at some of the National Council on Teacher Quality's seemingly ideologically-driven work (such as, I believe, its excessive focus on teacher pensions), much of its state policy analysis has a strong foothold in research and is one of the most comprehensive and regular analyses of state teacher policies. Like it or not, there is an increasing alignment between the NCTQ's scorecard and that employed by the U.S. Department of Education in the Race to the Top competition. The entire report should not be dismissed because of who they are (or are perceived to be). States should feel challenged by some of the analysis within the Yearbook and should consider looking to the "best practice" states identified under some of the metrics.

Here's a brief summary of the report's findings:

• State teacher policies are "broken, outdated and inflexible."
• Evaluation and tenure policies take too little or no account of classroom effectiveness. 47 states "allow tenure to be awarded virtually automatically."
  
• States are "complicit" on keeping ineffective teachers in classrooms. Only 1 state separates dismissal policy for poor performance from criminal and moral violations.
• Few states provide robust enough alternate routes into teaching.
• States' requirements for elementary teacher, middle-school teacher and special education teacher preparation are inadequate.
• There is too little accountability for teacher preparation in state policy. Only 5 states set minimum standards for teacher preparation program performance.
• States "cling to outmoded compensation structures," including the single salary schedule.
  

My primary quibble with the report is that it appears to completely and utterly discount the role of induction, mentoring and professional development in strengthening teacher effectiveness. Even if we prepare teachers better, recruit non-traditional candidates into the profession, retain them longer, compensate them differently, make evaluations more regular and meaningful, and find appropriate ways to terminate the small fraction of truly incompetent ones, it still will not be enough to maximize teacher effectiveness. There will continue to be a need for high-quality, individualized support upon entry into the profession and regular opportunities for data-driven, instructionally-focused professional development through a teacher's career. Professional development is not featured as a metric in the report at all and induction only enters as a criteria with regard to teacher retention, rather than teacher effectiveness -- which is where its most important power truly lies. That said, the evaluative criteria the report lays out about induction policy (on page 183-184 of the printed report) are worth noting and includes elements that states must attend to: mentoring of sufficient frequency and duration, mentoring provided at the start of the school year, and attentive mentor selection and high-quality training.

I won't beat this horse any further today, but check out these past posts for greater substance on what I'm getting at here with regard to the inadequate focus on the developmental needs of new and veteran teachers:

Race To The Top: Under The Hood
RttT: Redefining Teacher Effectiveness
Measurement Is Not Destiny

In other news, experts are doubting the likelihood of a 2010 reauthorization of the Elementary and Secondary Education Act, so these state teacher policies with an added dose of Race to the Top reforms is likely to be where it's at over the next year plus.

Using Value Added to Assess Teacher Effectiveness

The Association for Public Policy Analysis and Management -- an organization not widely known outside of academia and technical policy circles -- puts on truly meaty conferences. I've attended three APPAM conferences to date, including the Annual Fall Research Conference going on in Washington, DC this week.

Education is merely one strand at APPAM, but the sessions feature some of the biggest names in educational research addressing some very policy relevant issues. The current conference features sessions on value-added modeling, school choice, teacher certification and teacher induction, teacher performance pay, financial aid, college persistence, and more.

The session I attended yesterday on "Using Value Added To Assess Teacher Effectiveness" was excellent. It featured four papers each of which I will undoubtedly oversimplify in this brief blog post. (I encourage you to seek out the papers and read them closely -- below I've linked to those that are available.) One by Dan Goldhaber and Michael Hansen (University of Washington) suggests that year-to-year correlations in value-added teacher effects are modest, but that pre-tenure estimates of teacher job performance do predict estimated post-tenure performance in both math and reading. A second by Julian Betts (UCSD) and Cory Koedel (University of Missouri-Columbia) suggests that bias does exist in value-added models due to student sorting, but that it can be overcome through the use of multiple years of value-added data; further, the study suggests that data from the first year or two of classroom teaching may be insufficient to make reliable judgments about teacher quality. A third by Michael Weiss of MDRC suggests that that teacher variability carries implications for measuring program effects within randomized controlled trials when those teachers are not randomly assigned. And a fourth by John Tyler (Brown University) and Tom Kane (Harvard University) found that teacher assessments made using classroom observation rubrics (such as Charlotte Danielson's) are closely aligned with value-added ratings of teachers.

Superteacher To The Rescue!

Given the recent spate of federally-funded studies showing no effect of a variety of educational innovations and interventions, my predicted answer to the question ('Can Teachers' Talent Translate Elsewhere?') posed in this Houston Chronicle story is "no."

I worry, however, that the basic premise of the federally funded Talent Transfer Initiative is faulty and builds upon the notion of teaching (as reinforced by popular culture) as an individual rather than as a collective pursuit. Can 'superteachers' walk into dysfunctional school cultures and work magic that can result in a quantifiable impact on student learning? Some surely can. (It's too bad we can't clone Jamie Escalante and Frank McCourt, isn't it?) More important to ask is, should we expect them to?

What is more desperately needed than an expensive scheme to redistribute 'superteachers' is a serious attention to teaching and learning conditions. My New Teacher Center colleague, Eric Hirsch, spearheads assessment of school culture and the training of school administrators to more effectively shape it. His and independent research (here and here) has identified that teacher effectiveness is facilitated by a positive school context, including support from leadership, the existence of a collaborative working environment, and time for professional learning.

It doesn't appear that the Talent Transfer Initiative envisions teaching and learning conditions as part of the solution, and that's terribly unfortunate. I wonder if the TTI is even collecting such data to investigate the relationship between these variables and teacher success, or lack thereof? Until we address these contextual issues in low-performing and hard-to-staff schools, we're not going to get the results that we expect and students deserve.

UPDATE (9:35 p.m.) -- Claus von Zastrow offers an excellent blog post on Public School Insights about this study as well.

RttT: Redefining Teacher Effectiveness

My colleagues and I at the New Teacher Center have offered up what I believe to be a balanced and thoughtful series of recommendations to strengthen the teacher and principal effectiveness provisions in the U.S. Department of Education's proposed Race to the Top regulations. You can find the NTC's initial public comments -- submitted on August 21 -- here. And you find an addendum -- filed yesterday -- offering recommendations for specific language additions, here.

Generally, we are supportive of the overall direction of Race to the Top. But we feel that its focus on teacher effectiveness is too narrowly about measuring individual teacher impact at the exclusion of supporting all educators to strengthen their teaching and leadership skills and attending to teaching and learning conditions within schools that impact student success.

Here is a brief summary of our recommendations:

Improving Teacher Effectiveness and Achieving Equity in Teacher Distribution
• The RttT guidelines should include a definition of teacher effectiveness that acknowledges and
supports the development of teacher and principal practice, especially during the early years.
New teachers and principals, who disproportionately work in struggling schools, need strong
mentoring and support to become effective.

• The RttT guidelines should define ‘effective principal’ more expansively, drawing upon
additional measures of student success and data on teaching and learning conditions to fully
reflect the impact of teachers, school leaders, and school environment on student learning.

• The RttT guidelines should require states to address school leadership development and teaching and learning conditions in their strategies to improve teacher effectiveness and the equitable distribution of quality teachers.

Improving Collection and Use of Data
• RttT guidelines should specifically include teaching and learning conditions data gathered from
practitioners to help schools, districts and states better understand supports and barriers to
teacher effectiveness and equitable teacher distribution, and to incorporate this information into
their longitudinal P-20 data systems.

And here is some selected language that provides insight into our thinking around teacher effectiveness and teacher development:

Teacher effectiveness in the proposed RttT guidelines focuses exclusively on value-added student assessments. While value-added student achievement data can be used to reward and recognize certain achievements by educators, it should not be the sole method by which teachers are evaluated, observed, rewarded, and deemed “effective.” Firing the least effective teachers and rewarding the most effective alone is short-sighted and ignores the vast majority of teachers in the middle who can achieve greater success if given access to high-quality induction and professional development, strong and supportive school administrators, and opportunities for collaboration and leadership. Great teachers are made – not born. Teachers need professional support and opportunities to develop their practice, including focused induction during their initial years in the profession. It is important to measure teacher impact on student learning, but measuring impact without providing the means to help educators strengthen their practice will ultimately fail our schools.

If RttT is to be an effective reform strategy, it needs to recognize teacher development as a primary means to maximize classroom effectiveness. RttT should require states not merely to identify the best teachers, but see that their successes form the building blocks of a better understanding of effective teaching practice that can be replicated in classrooms across America.

And on teaching and learning conditions:

In order for school leaders to attract and retain quality teachers, research shows the need for school leaders to make decisions based on data that incorporate the perspective of classroom teachers. Teacher survey data can provide insight into the school culture, how decisions are made, and the use of instructional and planning time for teachers. Such contextual data may explain differences in teacher effectiveness between schools and districts. NTC has worked with over 300,000 educators in 10 states, and collected teaching and learning conditions data from over 8,000 schools to utilize in school improvement plans. In North Carolina, the State Board of Education now requires schools to utilize the data from the biennial working conditions survey to inform annual improvement plans and strategies.

Quality teachers will seek out and stay with strong supportive school leaders; therefore, using RttT funds for salary bonuses in hard-to-staff schools would not be the most effective approach. RttT should encourage states to show how they are using data from teachers, along with student achievement and other relevant data, to develop policies for these schools, strengthen school leadership, and ensure that they are settings where the most effective teachers want to work and can succeed.

The RttT public comment period closes today and a spate of organizations have submitted comments just under the wire. They range from narrow to broad, supportive to critical, and offer everything from research-based suggested line edits to what basically look like press releases buttering up Secretary Duncan.

Visit here to review all of the public comments submitted.