Modeling teacher effectiveness as a function of student ability

In 2010, the L.A. Times newspaper used the test results of Los Angeles County elementary students to assess and rank the elementary teachers. They then published the results on their website. Publicly ranking teachers in this manner has important implications on the careers of the teachers being ranked. It is, therefore, important that any model claiming to rank teachers be as accurate as possible. It seems plausible that a teacher's ability to help a student depends upon that student's prior academic ability. Some teachers might be better at teaching gifted students while others might be better at teaching remedial students. The L.A. Times did not account for this in their model. This paper looks at the results of allowing teacher effect to vary with prior student ability and how that interaction affects the relative rankings of the individual teachers. To assess this, the same Value-Added model the L.A. Times used is employed, with the exception that teacher effect is allowed to vary with the prior abilities of the students. New teacher ranks are then calculated and compared with the ranks calculated by the L.A. Times. The results of this analysis show a relatively small number of rank changes between the two models. In general, allowing teacher effect to vary results in a 5% to 12% change in the rankings of both the Math and Reading teachers relative to the L.A Times model. Other research on the same data has resulted in a 20% to 55% change in the rankings of the Math teachers and a 40% to 65% change in the rankings of the Reading teachers relative to the L.A. Times model. Although ranking teachers is a popular idea for determining the distribution of funding, the model shown in this paper as well as the other models reviewed, illustrate that a change in the model results in a change in the rankings of the teachers. A model that allows teacher effect to vary with prior student ability results in a better model fit than a model that does not. Whether or not this is a good thing is hard to say. Two examples are provided in this paper. One shows a teacher whose rank appears to be artificially inflated by this model and the other shows a teacher whose rank appears to be artificially lowered by this method. Although the fit of the model proposed by this paper is better than the model used by the L.A. Times, it does not result in radical changes in the rankings of the teachers. Rather, it seems that teacher rankings are sensitive to the particular model used and there are countless numbers of valid models. For this reason it is not wise to release such sensitive information to the public. It is probably true that the weak teachers are ranked relatively low in this analysis and that the truly good teachers are ranked relatively high. However, these rankings should only be used as one part of a larger metric to rank teachers and too much weight should not be placed on them for the purposes of rewarding or penalizing teachers due to the sensitivity of the model specification.