[tweetmeme source=”seprogram” only_single=false http://www.columbiasocialenterprise.wordpress.com%5D
By Luke Justice ’12
One of the most significant legacies of the No Child Left Behind Act (NCLB), passed under George W. Bush in 2001, is its impact on the way we use data to measure the performance of our educational system. While NCLB’s accountability measures where focused on school-level performance, 10 years later we are starting to develop teacher performance evaluations that, for the first time, incorporate student testing.
In 2009, The New Teacher Project published a now widely read survey of teacher evaluation systems across the country, titled The Widget Effect, and found that most systems were an extremely limiting and binary ranking of “satisfactory/unsatisfactory,” and that almost all (98 percent) teachers were annually ranked “satisfactory.” Even if there were a more accurate scoring in that old system, it would still tell us virtually nothing substantive about an individual teacher’s performance. Ten years after the passage of NCLB, some large urban districts are now developing systems that include the use of hard data in an attempt to paint a rich portrait of a teacher’s performance.
New York City is piloting a teacher evaluation system that uses a mix of student test scores and more qualitative measures, such as peer observations. Most of the focus on using student test scores to evaluate teacher performance has been on so-called “value-add” measurements that attempt to demonstrate the level of improvement students make under a particular teacher.
So far, the data seem to have been rather unreliable for the majority of teachers and more consistent at the extremes. That is, year-to-year, the value-add scores of exceptional — and exceptionally bad — teachers tend to persist, while those in the middle 80 percent jump around. Opponents of using student scores to rate teachers have latched onto this unreliability, while supporters note that the value-add measurements make up only one piece of the larger evaluation and are important data to have, especially if you are trying to identify the best teachers for recognition and the worst for improvement, if not removal.
Another significant drawback to value-add measurements is their limited application to a few grades and subjects. More complex material, such as essay-writing, can’t currently be captured by standardized tests. But the use of hard data to pin down an objective measure of student progress and thereby teacher performance may be just the first step in a larger cultural shift toward a more rigorous and integrated evaluation of both student and teacher. As The Widget Effect highlights, even districts with evaluation systems that have wider spectrums for ranking teachers suffer from a compliance mindset. That is, the most well crafted performance evaluations are useless if they are not applied with rigor and a genuine desire to understand one’s effectiveness.
As the culture around evaluations shifts, however, we are more apt to find methods for evaluating teacher performance that act as reliable predictors of student achievement in lieu of hard, standardized data on student outcomes. As but one example, Cincinnati has begun using rigorous teacher observations as part of its overall Teacher Evaluation System (TES). Researchers have found that the scores of these observations predict student achievement gains for subjects than can be accurately tested (e.g. reading and math), thus concluding that such observations can capture a significant portion of measurable differences in teacher effectiveness in harder to test subjects and grades.
It seems then that while data are directly relevant for understanding student achievement and teacher performance, the larger value of a focus on the numbers lay in the impetus it provided for a deeper cultural shift that seeks to develop an understanding of teacher effectiveness at all levels.
Continue the conversation at Friday’s Social Enterprise Conference session, Replicating Models of Success: Scaling Innovation in the Education Sector. Read a research report on Measuring and Creating Excellence in Schools.