The Use of Surveys in Evaluations: Evidence from the MET Study
  • By admin
  • February 11, 2020

By Grace Shelton

@GraceCShelton

Currently, 31 states use surveys in evaluations of teachers and principals. Previous research has shown that student, teacher, parent, and community feedback yields gains both in and out of the classroom. Teachers wanting to improve need to listen to their students, and principals should actively seek out staff feedback if they want an accurate measure of their performance. Our previous piece on state-wide survey usage examined which states use surveys and how NCTQ determines the value in survey data. Many of the research articles cited in NCTQ’s piece used evidence from the MET study on student surveys. A report from Bellwether Education utilized results from this study when comparing statewide evaluations, Chalkbeat used the study to predict whether or not a state would adopt surveys, and American University cited it when determining that students really can identify the most effective teachers.

In a 2014 study, Bellwether Education examined the use of surveys in statewide evaluations. The study discusses how each state takes a slightly different approach to survey data. For example, Massachusetts gives districts the flexibility to determine how survey data is incorporated and how much weight it should have. In Hawaii, survey data is vital: surveys and observations account for 50% of a teachers’ evaluation score, with the other 50% based on student growth and learning.

After Race to the Top funding gave states the motivation to revamp evaluation systems, the MET study, funded by the Gates Foundation, found considerable merit in the adoption of student surveys. As a result, many states began to adopt them into teacher evaluations. The study essentially concluded that students are capable of understanding whether or not a teacher is effective, and their feedback gives an accurate picture of teachers’ performance.

Bellwether urges readers to not find these results surprising, as students spend a considerable amount of time engaging with their teachers and “surveys have the benefit of aggregating the impressions of many individuals.” Bellwether concludes their study by saying that surveys add a view of teacher performance that test score data and observations simply can’t. In addition, student survey data is more actionable, and teachers can directly target specific areas to improve on. “Put simply,” the report concludes, “adding student surveys to…evaluation components generates a more reliable, robust, and actionable measure of teacher effectiveness at a relatively low cost.”

A 2012 Chalkbeat article also cited evidence from the MET study when examining whether or not New York would adopt surveys into their evaluation system (which they ultimately did not). The article states that, according to the MET study, student feedback and teacher observations were more closely related to teacher effectiveness than multiple other attributes or observations alone. At the time, the union was against the adoption of student surveys, worrying that it wasn’t appropriate to place such high-stakes decisions in the hands of students, and that teachers might seek out student approval more than student learning as a result. However, others stated that students are the only ones in the room, and their feedback could be used in multiple measures of evaluation. Ultimately, surveys were not used in formal evaluations, and New York, according to NCTQ, is the sole state that expressly excludes survey data.

In 2016, American University released an article detailing how students can identify the most effective teachers through their feedback. The article addresses concerns of student bias, and states that in well-designed surveys, students are never asked, “do you like your teacher?” Instead, students are asked how much they agree or disagree with a series of statements on teacher performance such as, “If I don’t understand something, my teacher will explain it in a different way.” In addition, results from the MET study led WAMU to report that teachers’ fears that if they were demanding too much of their students, they would get poor evaluations were unfounded. In fact, the more students reported a teacher challenged them, the higher the teacher was rated.

Survey data is used widely throughout the country. Much of this was encouraged by the MET study in their findings on student voice in teacher performance. As a result of this work, students are increasingly utilized in evaluations and, in many states, teachers are improving based on the direct feedback of those they teach.

 

High quality surveys are what we do at Resonant Education.  Get in touch with us to learn more.