Folk positivism

(or So there, Michael Bérubé! and other pseudo- philosophical rants)

With Michael Bérubé's challenge to his fellow nominees in the 2006 Weblogs Award Best Educational Blog category (hey, see that icon in the left margin? go vote for us now!), I'm going to ponder one of the problems I'm considering in Accountability Frankenstein. To wit, how the heck can we create accountability systems rooted in distrust of school officials and teachers (and students!), while simultaneously crafting systems that have a fairly blind trust of tests. That trust in test instruments undermines one of the main arguments for accountability, transparency. This is not a claim that psychometricians are absorbed in secrecy—well, not generally, though I understand Bill Sanders still keeps his value-added algorithms secret as a proprietary secret and plenty of state departments of ed fail to publish the psychometric data on tests, lest someone like Greg Camilli and Sadako Vargas discover the fudging that frequently happens in state testing systems—but an argument about the limits of test scores to tell us what children can do. In contrast to most other areas of expertise, the core purposes of school accountability are transparency and equity. Yet the routine of testing makes such knowledge virtually impossible, creating one or more layers of abstraction between what children do and what state governments say children do. That conflict has its roots in the development of professional expertise in testing early in the twentieth century. The end result of a century of the standardized testing industry is in tension with transparency. It raises a serious question whether testing technocracies are consistent with the democratic demands of accountability.

Others have written about technocracy in education. Art Wise's 1979 book Legislated Learning has the dangers of technocracy as one of its main themes, and Harry Wolcott's Teachers and Technocrats (1977) proposes that education is a moiety system divided into teachers and technocrats, if one looks at the creation and disintegration of efforts to apply the planning-programming-budgeting system (PPBS). And others have looked at technocracy more broadly, from Walter Lippmann's proposals that society be run by a small group of activists enlightened by neutral expert analysis to John Dewey's response (that we need the experts educating and informing the population, which can then become a Great Community) to the modern debates among folks such as Steven Brint, Frank Fischer, and Stephen Turner. There is a small but substantive literature on the growth of professions and the role of expertise-linked occupations in setting and implementing social policy. Based on it and the history of the testing industry, one might be tempted to attribute the role of tests to the growth of psychometrics as an occupation. And if we stopped in 1930, that supposition might be true.

Yet there is something a little different about the worshipping of test scores; it is less the power and authority given to psychometricians than the social assumptions made about tests. As Haney, Madaus, and Lyons wrote in The Fractured Marketplace for Standardized Testing (1993), a variety of market and political pressures has led to the corrupt use of tests (e.g., one NYC principal who tried to decide on valedictorians via SAT scores), so much that trained assessment experts casually acknowledge that the uses of standardized tests commonly fall outside their validated purposes. Some trained in assessment are the harshest critics of much modern standardized test use (Bob Linn is probably the shining star in that constellation), and yet their voices are largely ignored. Why?

Martin, Overholt, and Urban's Accountability in American education: A critique (1976) provides an alternative explanation: it is not so much in the professionalization (which they didn't discuss) as the intellectual history of positivism that leads accountability advocates to believe that test scores are concrete and consequences will drive improved instruction in a behaviorist manner. In referring to Auguste Comte and 19th century positivist arguments, the authors assert that there is a direct line from Comtean positivism to the uses of testing in the 1970s and Leon Lessinger's 1970 arguments for accountability.

While I understand the appeal of this argument, and I've seen similar arguments in other contexts, I am less than convinced, for both theoretical and evidentiary reasons. Theoretically, it is perhaps a mild inconsistency for postmodernist critiques of positivism to assert a linear intellectual path for anything, let alone a genealogy of contemporary beliefs in the utility of statistics. I'm not saying that Martin, Overholt, and Urban were being all postmodernist 30 years ago, but since they're using post-Romantic continental philosophers, there is this guy named Nietzsche who talked about the problems with asserting linear intellectual legacies for anything. Second, it's hard to discern where Comtean positivism would be more influential than American pragmatists, who (Dewey among them) were confident that this new thing called science could be used to create beneficial social policies.

But theory isn't the real reason why I'm skeptical of this linealogy. Fundamentally, it requires considerable faith to believe that politicians and others who advocate high-stakes accountability are well-read in 19th century philosophy, know that much of behaviorist literature, or even can put forth anything more than bluster to argue the utility of test scores as a direct reflection of reality. What we see today is a folk version of positivism, the rough-hewn confidence that test scores must "mean something." Arguments about the construction of social problems don't really help us here, because this is the classic case of ideas diffusing in different directions without any Powers That Be giving them direction. In short, I'm not sure we have a way to wrap our heads around folk positivism. But I think we need to.