Education Opportunity Network

Education Opportunity Network -

Are Teacher Evaluations Education ‘Reform’s’ Biggest Bust?

Would you like your job performance judged by a five-year-old?

That’s a relevant question for public school teachers in Hawaii, where the state’s new teacher evaluation system attributes ten percent of their job performance rating on what children as young as 5 years old think.

Although ten percent may not seem like a whole lot, in a metric based evaluation system where harsh judgments of “effective” versus “needs improvement” can swing either way based on a point or two, ten percent can be one hundred percent of the reason for a bad grade.

But the child’s portion is not the sole problem Hawaiian teachers are having with their new evaluation system, which will ultimately affect their pay and can subject them to penalties as severe as termination.

As the article cited above reported, a recent survey conducted jointly by the state Department of Education and the teachers’ union found that “as many as four in five” teachers responding to the survey have problems with the new evaluations, ranging from “confusion … to skepticism about its fairness”.

Hawaii isn’t the only state having problems with new teacher evaluation systems that are being rolled out across the nation at the encouragement – others would contend, coercion – of the federal government.

According to Education Week, at least a dozen states have asked the U.S. Department of Education to allow them delays in rolling out new teacher evaluations systems. Two of those states, Maryland and North Carolina, received Race to the Top grants that committed them to erecting new teacher evaluation systems. The rest of the states pledged to implement new evaluation systems in order to receive waivers from the federal No Child Left Behind law.

In states that have had more success at implementing new teacher evaluations, the results have been decidedly underwhelming.

As Education Week reported last year, “In Michigan, 98 percent of teachers were rated effective or better under new teacher-evaluation systems recently put in place. In Florida, 97 percent of teachers were deemed effective or better. Principals in Tennessee judged 98 percent of teachers to be ‘at expectations’ or better last school year, while evaluators in Georgia gave good reviews to 94 percent of teachers taking part in a pilot evaluation program.”

Indiana‘s new evaluation program found, “88 percent of teachers and administrators were rated as either effective or highly effective under the system; only about 2 percent need improvement, and less than a half a percent were deemed ineffective.”

In many of these states where supposedly under-performing teachers have been spotted, there are numerous anecdotes that the labeling has been either highly questionable or blatantly mistaken. Teachers in Florida, for instance, have their performance rated using the test scores of students they’ve never even taught. Really!

More recently, in Washington state, the stakes over new teacher evaluations ratcheted up further when legislators refused to commit the state to new evaluations, which could result in the state losing control of over roughly $38 million in federal funds for schools serving low-income students.

And just this week, a key underpinning to the whole teacher evaluation program pushed by the Obama administration was cast into doubt. As Education Week’s Stephen Sawchuk again reported, the American Statistical Association, “the world’s largest community of statisticians,” examined the practice of basing teachers’ performance evaluations on students’ standardized test scores – a key criterion for getting Race to the Top money or an NCLB waiver – and warned against this approach.

All this controversy over teacher evaluations could be just a grand debate among academicians and policy wonks if it weren’t for the fact that these new schemes are doing great harm to teachers and, consequentially, the students in their charge.

Further, results from these questionable measures are being used to construct all sorts of new mandates, building an even more imposing policy edifice on a foundation of sand. While in the meantime, the only response from those in charge has been to “stay the course.”

How We Got Here

The idea of basing the fate of teachers, even whole schools, on how students perform on standardized tests started as a theory cooked up by what University of Texas education professor Julian Vasquez-Heilig has called “a motley alliance” between civil rights proponents and politically influential foundations and organizations who advocate for more private sector control of public education.

Civil rights proponents have long maintained, correctly, that schools that serve the most disadvantaged children have tended to have the least qualified teachers, as measured by preparation, experience, and other factors. So in the interest of “equity,” these advocates insist that in order for states to receive grant money from big ED, or have certain federal laws waived, they must impose teacher evaluation systems that use standardized test scores to evaluate teacher “effectiveness” and then distribute the “most effective” teachers more widely to under-served schools.

The goal of privatization proponents, on the other hand, is to reduce the role of the state in public education and shift the education system towards a profit-making enterprise. Labeling teachers effective/ineffective serves their purposes because when they can define the act of teaching as an “output” – a rating based on student test scores – they can use as leverage to pry teachers’ unions away from their influence and argue for giving pay raises and merit rewards to smaller pools of only the most “effective” teachers.

Any attempt to ease the impractical and often erroneous mandate to evaluate teachers based on student test scores is met with strong opposition from these two seemingly incompatible parties.

With the backing of what appears to be a “bipartisan” constituency, lawmakers and policy wonks line up to support  these evaluation systems, even though the technical aspects are largely unresolved.

Reducing Teaching To A Math Problem

Reflecting on the new ASA study referenced above, education journalist Valerie Strauss wrote on her blog at The Washington Post, that current teacher evaluation methods of evaluating teachers “purport to be able to take student standardized test scores and measure the ‘value’ a teacher adds to student learning through complicated formulas,” but “these formulas can’t actually do this with sufficient reliability and validity.”

As Rutgers professor Bruce Baker has stated on his School Finance 101 blog, “different choices of statistical model or method for estimating teacher ‘effect’ on test score growth … even subtle changes … can significantly change individual teacher’s ratings and significantly reshuffle teachers across rating categories.”

Does that sound like a valid and reliable evaluation to you?

Yet in the meantime, politicians and policy makers are advocating for these evaluatons – measures that they by-and-large do not understand, as Kevin Welner of the National Education Policy Center recently observed. Writing on the blog site of education historian Diane Ravitch, Welner explained, “The math is just too complex … vectors capturing the effect of lagged scores, mathematical descriptions of Bayesian estimates, and within-student covariance matrices … has the obvious effect of placing policy makers at the mercy of whichever experts they choose to listen to.

Welner concluded, “We should, at the very least, recognize and acknowledge the reality that these policies are being adopted by policy makers who pretty much have no clue what it is that they’re putting in place.”

Consequently, the results on the ground rarely resemble the neat and clean explanations given to lawmakers by the “experts.” In Connecticut, for instance, the new teacher evaluation system has resulted in something “conceptually appealing,” according to David Title, superintendent of schools in that state, but “very difficult to do technically … There are so many different variables that impact student achievement … What you’re not able to do, in my view, is prove cause and effect.”

The Impact On Teachers

With so little understanding of the technical difficulties with teacher evaluations, it’s even more important to consider, as Welner argued, “the non-technical evidence” of teacher evaluations.

Indeed many of the non-technical results of new teacher evaluations should make their use questionable to anyone with an ounce of common sense.

In Hawaii, teachers are understandably concerned that “children as young as 5 years old who evaluate them will put ‘thought and effort” into their answers.'” The data input for the system “obliges teachers to prioritize testing over more constructive forms of teaching,” according to one veteran teacher. And more time that teachers spent on instruction has been diverted to filling out paper work.

In Connecticut, school principals have to complete 17 reports throughout the year for every single teacher in their school, and teachers have to spend hours completing evaluation goals, student learning targets, and observation records – all of which take time from educating students.

In Rochester, N.Y., teachers are suing the state because the new evaluations don’t take into account any student factors such as poverty levels and number of absences.

New Jersey teacher-blogger Jersey Jazzman recently observed that the proposed evaluation system for his state “relies on Student Growth Percentiles (SGPs), based on standardized tests, to evaluate teachers. Yet the very man who is the “inventor” of SGPs has said that they cannot be used to determine a teacher’s effect on student learning!” (emphasis original)

As Diane Ravitch observed in one of her numerous blogposts about faulty teacher evaluations, “My government spent billions to find teachers to fire, and all we got was confusion.”

The Benefits Of All This?

Meanwhile, no one advocating for test-based teacher evaluations can be satisfied.

Those who advocate for teacher evaluations that result in distribution of more high-quality teachers to schools serving low-income minority students don’t have much to celebrate. Tennessee, for instance, has had a teacher evaluation system based on student test scores in place longer than most other states. When that state outpaced the rest of the nation in gains on the most recent National Assessment of Education Progress, some credited it to the state’s teacher evaluation system, among other “reforms.”

However astute blogger Audrey Amrein-Beardsley, who devotes her site to the issue of teacher evaluations looked at Tennessee’s accomplishment more closely and found the state has an expanding achievement gap. “The state’s lowest socioeconomic students continue to perform poorly on the test.” Tennessee didn’t make gains significantly different than many other states. And “other states with similar accountability instruments and policies (e.g., Colorado, Louisiana) did not make similar gains, while states without such instruments and policies (e.g., Kentucky, Iowa, Washington) did make similar gains.”

Teachers in Tennessee, in fact, have filed two lawsuits against the state for its unfair evaluation scheme.

For those who want to see a greater role for the private sector in public education, the results are frustrating too. Writing at the conservative journal National Affairs, Frederick Hess of the American Enterprise Institute recently blamed this colossal policy failure on “getting public employees to actually do what policymakers think they’ve told them to do.”

Insisting, “the right response to these disappointing trends is certainly not to abandon the reform agenda,” Hess declared, given his observation that “schools and districts do not go out of business,” a need for “a complete reform movement” that is “willing and able to rethink old norms.”

Certainly, one of those “old norms” is that the quality of teachers matters a great deal to our students. But it makes no sense to believe we’ll get better quality teachers by treating them this badly.

Even a five-year-old could see that.

  • Linda Strauss says:

    If those who want to destroy public education are able to set into motion tests and policies that “prove” the ineffectiveness of public education, they have succeeded.

    April 16, 2014 at 6:18 pm
  • ANTONIO BERNAL says:

    TEACHER EVALUATION IS PURE POLITICS. I was a teacher in the Los Angeles school system

    for 23 years, and as such I was able to evaluate the evaluators.
    Since I had been in Cuba several times (I was a Spanish teacher), and spoke openly about

    it, there were teachers and and administrators who simply hated my guts. We would pass

    in the hall and they refused to greet me. Then I began to notice unannounced “visits”,

    sometimes more than one a day, where the administrator walked about and looked at

    everything, and even asked the students if they understood the classwork. Every single

    time the class was concentrating and quietly on task. Since my formative years were

    spent outside of the US, my classes were a place of genuine culture and language

    development. I never treated Spanish as a foreign language, nor as something for

    tourists. We studied the classics, from Cervantes to Calderon to Sor Juana (in Spanish,

    of course). We studied Garcia Lorca, and the Latin American boom- from Garcia Marquez to

    Borges to Fuentes. I am talking about High School with a 90% Latino population. I had

    the students work collaboratively, where they enthusiastically got together and wrote

    movie scripts. Then we would go to someone’s house or the park they would act out the

    movie, which I filmed and my son edited. They got together to make oral reports in front

    of the class. We had raging arguments and debates from religion to abortion or anything

    in between. I remember a girl was so passionately anti-abortion and defended her

    position so eloquently, I gave her an A. The point was not to have the “right” opinion,

    but rather to use the language as an effective tool. Then I noticed further, in the last

    year or so, that I was getting odd students in the middle of the semester. One had a

    police ankle thing. Another was a retarded girl who pulled the top of her dress so low

    as to almost show her nipples. She accused me of telling her how to avoid getting

    pregnant, presumably because I was planning a come-on. Fortunately the administrators

    decided they didn’t want to get into that hornet’s nest, although I was constantly told

    that I was responsible for everything, because I was the “authority”. Then there was the

    young man who never sat down, never turned in a paper, but would spend the hour talking

    to every student in turn. I went to see his parents, but they accused me of attacking

    them (figuratively speaking). Little by little I stopped teaching Spanish and began to

    spend my time on discipline, on warehousing the avalanche of students with serious

    mental problems that were assigned to me. I eventually got the message. Teaching had

    been a joy, and they had turned it into a chore. I handed in my retirement papers, even

    though I was not at the top of the scale, and was forced to retire with a smaller

    pension than what would account for my time there. I had taught in another country,

    where the advancements on the pay scale were automatic. Here the union rep had never

    explained how the retirement system works, and I found out too late.

    April 16, 2014 at 7:15 pm
  • Carl R. Carnein says:

    Ineffective administrators are the bane of teachers in the US. Administrators should provide a nurturing environment and autonomy. Instead, they seem to subscribe to the “one size fits all” mentality that has taken over, thanks to crappy college programs. Administrators should be required to teach for “x” years before being allowed to take over school operation.

    April 16, 2014 at 7:22 pm
  • jimmy midnight says:

    I’ll read this post more carefully a little late, because Jeff’s stuff is always worthwhile. I just want to add, while I’m thinking about it, the hostility to unionism, up front among Republicans but also practiced by many Democrats when they think no one’s looking, that is an important component of the dizzy drive for “teacher evaluations.”

    April 16, 2014 at 11:10 pm
  • Estelle Dahl says:

    Never let facts get in the way of your ideology. That’s the mantra of too many politicians and their misguided supporters. Others are just hellbent on privatizing our public schools for their own financial gain. It’s a terribly sad time to be a teacher and/or believer in public education. I sure hope the public wakes up soon before we have entirely lost the public school system that took us to the moon, and beyond!

    April 17, 2014 at 5:35 am
  • paulpnasutiPaul P. Nasuti says:

    With a bi-polar thinking tool, man has become habituated to reacting to what is wrong, as one often must. Human nature allows this to be so. Therein lies the genetic bottom line. But we do not grow up and develop a vacuum. In response to that inherent condition that requires a contrasted stimulus field in order to gain new insight (learn), we utilize the accumulated wisdom of the world around us, beginning with mother, family, community and culture, to add to innate intellectual sklls. With stimulus input of others we learn languge which amplifies our born in need to learn. Our present form of learning, Institutional Education, is stuck in place with an obsolete model that continues to perpetuate reactive response to that which is wrong. We need a systemic shift to improve our ability to think outside the box by employing an Earth based education model, able to support the growth and development of the whole child, rather than just the part that can be measured. All of the arguments above, whatever their virtues, ignore the common ground that has grown from the habit of responding negatively to each specific need. Without abandoning this need, we also must begin discussion of a responsible, adult, pro-active shift to change the big picture. The balanced, win/win answers we seek lie in the middle ground, both logically and genetically, and are amply exampled in the natural world around us. We are seven billion and counting. Continuing status quo behaviors has a limit. We know what happens to lemmings in an ecosystem overburdended by its own population.

    April 17, 2014 at 7:27 am
  • Ken Mortland says:

    Washington state will not “lose” money because of the legislature’s vote and I’m surprised you don’t know that. What is lost is the flexibility of use, as the funds become subject to federal use mandates and are no long open to more flexible applications.

    April 17, 2014 at 7:32 pm
    • Jeff Bryant says:

      Ken, thanks for reading and commenting. Here’s what I said (emphasis added): “result in the state losing control of over roughly $38 million in federal funds.” That’s not the same thing as “losing,” which is how you read it. So I think you and I see the potential consequences the same way.

      April 17, 2014 at 7:43 pm
  • Mary Krotki says:

    So very well written! Now someone please do something about this!!! I have less and less time to teach or prepare! Students are ultimately the ones who suffer through all of these so called great ideas! The evaluation forms have changed so many times (let’s waste more paper and ink!) that WE are supposed to now, at the end of the year, try to figure out what tests that we’re being evaluated on since the ones that we were told to pick for our end of the year evaluation are no longer being used. Why should teachers have to rewrite evaluation papers that we already wrote and rewrote numerous times?!? The whole process is absurd and a waste of time! Maybe everyone in every job should have to start writing several evaluation papers to prove if they’re doing their job or not; let’s start with the people who want this to be implemented!

    April 20, 2014 at 3:16 pm

Leave a Reply