US Magistrate Judge Stephen Smith on May 4 handed Houston public school teachers a victory when he allowed a suit against the local school district’s questionable teacher evaluation system to go to trial.
The judge said that the suit by seven teachers and their union, the Houston Federation of Teachers (HFT), raises valid concerns about their employment rights.
HFT in 2014 filed the suit to stop the Houston Independent School District from using EVAAS, an automated system managed by a private contractor, to evaluate teacher performance.
The union filed the suit on behalf of seven high performing and award-winning teachers who received bad performance evaluations from EVAAS.
EVAAS is a value-added measure (VAM) software system marketed by its owner SAS as an analytical tool that measures the value that teachers add to their students’ education.
The software supposedly measures value added by using a secretive and proprietary algorithm that compares student standardized test scores over time.
The union’s suit describes EVAAS’ evaluation methodology as “complex and opaque.”
A 2014 study by education experts found that value-added measurement was not an effective way to evaluate teacher performance, but HISD as well as at least 30 states continued to use it.
HISD had sought a summary judgment to have the union’s suit dismissed, but Judge Smith on May 4 ruled against the district and allowed the suit to proceed.
“Houston developed an incomprehensible, unfair, and secret algorithm to evaluate teachers that had no rational meaning,” said Randi Weingarten, president of the American Federation of Teachers. “Judge Smith saw that it was seriously flawed and posed a threat to teachers’ employment rights; he rejected it. This is a huge victory for Houston teachers, their students, and educators’ deeply held contention that VAM is a sham.”
According to Audrey Amrein-Beardsley in her blog Vamboozled, VAM was developed by a statistician with the US Agriculture Department named William Sanders who thought that education achievement and teacher performance could best be evaluated by a statistical model similar to one he used to measure the development of animals.
He sold his VAM rights to SAS, a private company that sells software and data management consulting services to businesses and government agencies.
SAS used VAM to create EVAAS, which it then sold to school districts around the country that were trying to comply with the 2002 No Child Left Behind law that, among other things, encouraged states and school districts to use student standardized test scores to evaluate teacher performance.
At a cost of $500,000 a year, HISD in 2011 contracted with SAS to evaluate teacher performance by using EVAAS.
(Last year, the HISD Board of Education voted not to renew its evaluation contract with SAS.)
Judge Smith noted in his ruling that using a private contractor to evaluate teacher performance waiproblematic because SAS treats EVAAS as a trade secret and won’t divulge how the software evaluates teachers.
The union’s suit alleges that the secretive nature of EVAAS and VAM make it impossible for teachers who receive a poor EVAAS evaluation to challenge the evaluation, thus depriving them of due process.
“Due to a faulty, incomprehensible, and secret formula, good teachers are being labeled failures,” said Weingarten in 2014. “We have enough evidence to make clear that not only has VAM not worked, it has been really destructive and it in no way helps improve teaching and learning.”
At about the same time that the union filed its suit in 2014, Dr. Morgan Polikoff, assistant professor of education at the Rossier School of Education, University of Southern California, and Dr. Andrew Porter, dean and professor of education at the Graduate School of Education, University of Pennsylvania, published a paper questioning the effectiveness of VAM.
Polikoff said that there is little evidence to show that VAM is a reliable method for evaluating teacher performance.
“While value-added measures do provide some useful information, our findings show that they are not picking up things we think of as good teaching,” said Polikoff, whose research was funded by the Bill and Melissa Gates Foundation.
Given the extent to which VAM is being used, Polikoff said that his and Porter’s findings were “troubling.”
“Our results suggest that it’s going to be difficult to use (VAM-based) systems to improve teacher performance,” one of the selling points that SAS uses to market EVAAS.
Given the evidence against EVAAS and VAM and Judge Smith’s ruling, Zeph Capo, president of HFT, called on the school district to take immediate steps to right the wrong done by EVAAS.
“With (Judge Smith’s) decision, Houston should wipe clean the record of every teacher who was negatively evaluated,” said Capo. “From here on, teacher evaluation systems should be developed with educators to ensure that they are fair, transparent, and help inform instruction, not be used as a punitive tool.”