The Role of Surveys in State Evaluations
  • By admin
  • February 5, 2020

By Grace Shelton

@GraceCShelton

 

The National Council on Teacher Quality (NCTQ) recently released the first report in a series of three on tracking states’ teacher policies. This particular report focused on evaluation policies for teachers and principals. This report is particularly important in that it places all 50 states and the District of Columbia in a comparison that allows us to pull best practices and examine trends. NCTQ’s report, and its supporting research, resonates with us because the analysis focuses considerably on the use of surveys in state evaluations.

Using surveys in evaluations of teachers and principals has met some criticism over the years, but NCTQ finds that not only do surveys provide a more reliable and valid measure of quality, but of those states that implement surveys in evaluation, all have met teacher and principal effectiveness goals. Not only is NCTQ unique in their access to state-level data, but because these reports are conducted annually, NCTQ has the opportunity to put this year’s state data in the context of data from years prior. Therefore, we can see the trajectory of survey usage in evaluations and determine whether or not there is an increase in states’ incorporation of surveys.

Surveys in Teacher Evaluations

The report examines how states incorporate, or don’t incorporate, the use of student feedback in teacher evaluations. In other words, whether or not states require or encourage student surveys as a measure of teacher quality. Previously, teacher evaluations focused primarily on observation data and test score achievement to get a holistic view of teacher quality. However, research has been showing that in order to truly understand how a teacher performs, student feedback must be gathered. This makes a lot of sense, especially considering how students have thousands of hours of observation time with their teacher, compared to the smaller amount of time an external observer would have access to.

However, teachers have often had apprehension when it comes to student surveys. Some teachers worry scores will be based on teacher popularity or likability more than skill. Additionally, teachers worry that if they challenge their students in the classroom, this might also hurt their scores. NCTQ finds that when surveys are well-designed, they can “function as a very important part of evaluation.” Not only that, but the report also explains that survey feedback correlates with student learning gains, “providing schools with another independent source of teacher performance, alongside state tests.”

In their observations, there are currently 31 states that either require or allow surveys to be used in evaluation scores. This is a slight decrease from 33 states back in 2015. States, according to the report, can choose to opt out of surveys for a “myriad” of reasons, but all in all, surveys collect vital information about an educator’s performance.

Surveys in Principal Evaluations

When it comes to principal evaluations, the use of surveys doesn’t appear to extend to student feedback. Instead, in order to measure principal effectiveness, teacher, parent, and community feedback is often gathered through surveys instead. In the particular case of principal surveys, the report found that using surveys provides a “more complete picture of performance than observation data alone.” Currently, 22 states allow principal surveys in evaluations and 9 states require it. This means that 19 states have no policy around principal surveys and 1 state expressly prohibits them.

Principals, as we’ve discussed in previous blog posts, have an incredible amount of exposure with students, teachers, parents, and the school community. As a result, they have a wide array of responsibilities. It is difficult to evaluate principals along the same scale, as there is a significant amount of context that goes into every principals’ job performance. As the report states, “…given the significant variation in principals’ effectiveness and the impact strong principals have on student achievement, in-school discipline, parents’ perceptions of schools, and school climate, survey data can add important context to principal evaluations.”

States’ Success with Surveys

So, we know that 31 states are currently requiring or encouraging student surveys in teacher evaluations and 31 states are also requiring or encouraging staff/parent surveys in principal evaluations. NCTQ compiled a state-by-state summary of differing policies and analyzed whether or not states met respective goals and what systems they have in place to help them meet said goals.

In regard to principal effectiveness, there were 11 states in 2019 that met their respective goals. Of these states, every one of them either required or highly encouraged the use of surveys and feedback in principal evaluations. These states were Colorado, Connecticut, Delaware, Florida, Hawaii, Massachusetts, Minnesota, Michigan, Pennsylvania, South Carolina, and West Virginia.

Additionally, there were 5 states that met teacher professional growth goals. These states were Colorado, Indiana, New Mexico, Ohio, and South Carolina. Again, each and every state that met their goal either required or encouraged student surveys in teacher evaluation.

These results show that there might be a connection between the use of feedback outside of formal observations and ultimate effectiveness. Survey data, when incorporated into evaluations, provides invaluable data to evaluators and teachers/principals who seek to improve their practice.

Research Used

The report uses a myriad of research to explain the importance of effective evaluations. However, there are a few pieces of work that focus solely on survey usage and shed light on the measured benefits of student feedback in particular.

Firstly, the report cites a 2013 research article that measured the effect of student feedback on three dimensions of teacher quality: classroom management, cognitive activation, and supportive climate. The longitudinal study confirmed that student ratings are “useful measures of teaching quality in primary school” and that students’ ratings of teachers across all three dimensions were either able to predict student achievement or students’ development of academic interests.

Additionally, the report cited another 2013 research article that stated that although the construct validity of student perceptions of teacher quality isn’t perfect, it is still relatively high. The research stated that student surveys are generalizable in terms of structure and classroom management ratings, but motivation, understanding, and student involvement results were mixed.

Lastly, the report uses findings from a 2016 article that specifically examined the popular Tripod Student Perception Survey. This survey, the primary focus of a commonly cited MET study, suggests that class engagement and management were all positively associated with higher survey scores.

Recommendations

NCTQ concludes their report on teacher and principal evaluations with a list of recommendations based on best practices in highly successful states. NCTQ urges states to use objective measures of student growth to improve validity of evaluations and to conduct multiple observations. In addition, the report suggests eliminating binary evaluation systems so that there is a better sense of just how effective a teacher or principal is.

However, the final and perhaps key recommendation states that “survey data provide important information about an educators’ performance”, indicating that although not every state utilizes surveys in evaluation, and research can be a bit mixed, based on the success stories and best practices of others, surveys should be used in evaluations.

 

Visit Resonant Education to learn more about high-quality Educator Effectiveness surveys.