Test Score Incentives Aren’t Fair

By Michael Diedrich, Policy Associate, February 10, 2012 at 11:30 am

We've already looked at how linking teacher incentives like pay or retention to test scores misunderstands the theory of motivation and doesn't actually improve results. As our state debate about seniority continues, however, we're seeing plenty of arguments that are more about fairness than they are about the realities of effectiveness. Unfortunately, the leading metric likely to replace seniority is test score data, which isn't a fair system, either.

Let us remember that seniority itself was first introduced to increase fairness in teacher retention. Since experience often correlates with teacher quality, seniority was an objectively measurable proxy for quality that could protect teachers from the whims of administrators.

Many now question whether seniority really is the best standard for determining teacher retention. They cite examples of promising young teachers who are let go while older, less effective teachers are retained. In the quest for a new objective proxy for quality, we seem to have settled on test scores.

The thing is, test scores haven't turned out to be any fairer than seniority. As just one detailed example, check out this report [PDF] from the American Educational Research Association (AERA) that takes a close look at Houston, Texas, where a test score system has been used to assign bonuses and determine teacher retention for a few years now.

Consider the story (cited in the report), of an elementary teacher with 10 years of experience – not a calcified “lifer” – who had in the past been celebrated as a Teacher of the Year. After being shuffled between fifth, fourth, and third grades, her “value-added” measurements on state tests fluctuated dramatically and got her fired.

How is this possible? It turns out that test scores, despite their shimmer of objectivity, aren't particularly reliable and are nowhere close to a reflection of any individual teacher's performance. When looking at San Diego and four different counties in Florida, the AERA report found that seven or eight out of every ten teachers in the bottom 20% of the test score distribution wasn't in the same range the next year. One in four of the bottom 20% ended up in the top 40% the next year. The AERA report cites many other examples and studies confirming testing's unreliability.

Test scores don't look any fairer than seniority, and we deserve answers to tough questions about what's going to replace seniority before we abandon it.

Posted in Education | Related Topics: Teachers  Teacher Assessment