Measuring Teacher Effectiveness: Credentials Unrelated to Student Achievement

This study that took 2 decades of information from the Florida school system disclosed the following:

While not mentioning unions, the study refutes most of the evaluation techniques and seniority rules set forth in most union contracts.

By stating that it is imperative to weed out the chaff prior to tenure being granted, it also makes a case for eliminating tenure.

Money is not the most important factor in student learning, the quality of the teaching and the teacher is. Our Federal government thinks that it can dictate minimum test scores and throw money at the system and improve student performance. This study refutes that hypothesis.

Excerpt: The Florida Study

As with most previous research, we found no relationship between a teacher’s earning a master’s degree, certification, or years of experience and the teacher’s classroom performance as measured by student test scores. Though we found that some pedagogy course work was related to teaching effectiveness, the magnitude of the effect was mild: even very detailed information about the teacher’s preparation in college told us very little about how effective that teacher would be in the classroom.

The Current System

In making decisions on pay, promotion, and tenure, U.S. public schools today do not seriously consider measures of how well a teacher performs in the classroom. Instead of distinguishing between the observed performances of teachers, the current system differentiates teachers by the number of advanced degrees that they hold and their years of experience in the classroom.

Teachers must acquire a certification, the vast majority of which are gained by graduating from a college of education. Once a candidate earns a teaching position, however, that teacher no longer undergoes serious evaluation. The evaluation system relies on infrequent and superficial observations of a teacher’s performance, often during a single class period (or less) at a time known by the teacher in advance. Consequently, nearly all teachers are classified as effective: an analysis of 12 large school districts in four states by the New Teacher Project found that in districts using a binary evaluation system (i.e., with only two ratings: “satisfactory” or “not satisfactory”), over 99 percent of teachers received the thumbs-up rating. Even districts that used broader evaluation distinctions ranked 94 percent of teachers in one of the top two tiers of effectiveness and deemed just 1 percent “unsatisfactory.”[1]

The vast majority of public school systems pay teachers uniformly, according to a salary ladder built on only two key factors: advanced degrees and experience. In American public schools, the single-salary schedule has a long history that actually predates the rise of teachers’ unions as a political force: as early as 1918, about 65 percent of American urban school systems reported that they had established a salary schedule.[2] Today, single-salary schedules based entirely on these two factors are the norm nationwide.

Conclusion:

Modern research on teacher quality makes clear that the factors used to determine a teacher’s compensation tell us little to nothing about how well the teacher will perform in the classroom. That consistent finding has (or should have) enormous implications for the future of the current system. The results of an employment policy based entirely on credentials uncorrelated to student achievement are obvious: we see wide variation in the quality of public school teachers.

The structure of the current system is simply indefensible, given modern research findings. There is nothing inherently wrong with relying on proxies for effectiveness when making employment decisions. However, when those proxies fail to differentiate meaningfully between the most and the least productive workers, they should be jettisoned. This is certainly the case with our public schools, where wide variation in teacher quality persists among those who have passed through the usual screens and earned the recommended degrees.

Just how much does teacher quality vary? An early study by Stanford University economist Eric Hanushek estimated that the difference between being assigned to one of the system’s best teachers and one of its worst is about an additional grade level’s worth of proficiency at the end of the school year.[6] Similar variations in teacher quality have been found in Tennessee,[7] New Jersey,[8] Chicago,[9] Florida,[10] and other unnamed school districts across the country.

Most parents, teachers, and school administrators surely won’t find it surprising that teachers are not identically effective. What to do?

The findings of our research and other studies suggest that public schools should revise their employment systems so that they prioritize measures of the teacher’s actual effectiveness in the classroom. School systems should develop comprehensive evaluation systems that utilize quantitative (e.g., test scores) as well as qualitative (e.g., classroom observation) measures of teacher effectiveness. The results of these evaluations should be used to determine which teachers are retained in the classroom, how much a teacher is paid, and whether the teacher receives job protections in the form of tenure.

Though there is much work to be done, several school systems have begun developing and adopting rigorous evaluation systems for their public school teachers with the hope of using their results to inform employment decisions. Over the last several years, researchers have used variants of a technique known as “value-added” analysis to study the extent to which teacher quality varies. Essentially, this procedure measures how much an individual teacher contributes to his or her student’s test scores after accounting for factors that the child brings into the classroom. Research utilizing the value-added approach consistently finds that teacher quality is the most important factor within a school’s control for boosting student performance. Students with nearly identical backgrounds will perform quite differently on standardized tests, depending on the teacher to whom they were assigned.

Taking into account measures of a teacher’s effectiveness has an enormous effect on the public school system. For instance, New York City’s use of a new evaluation system when making tenure decisions this year led to a dramatic decrease in the number of teachers receiving those job protections. School systems across the country should continue moving toward emphasizing direct measures of teachers’ performance, rather than credentials, to identify and compensate the best teachers.

Over the last two decades, we have learned two important lessons about public school teachers: teacher quality varies dramatically; and almost nothing we know about a teacher before he or she enters the classroom accurately predicts how successful that teacher will be. Now heavily documented through empirical research, these findings should point us toward a fundamental transformation of our system for evaluating public school teachers.

Read full Manhattan Institute report here.