A teacher’s impact cannot be measured… or can it?

Although the system is facing immense challenges in teacher retention and recruitment, we need more than just an adult in the room. Rightly, we have invested considerable time and resources in developing effective teachers, but we cannot know the impact of this investment without a reliable measure of teacher impact.

Developing such a measure has been a longstanding challenge. Superficial quantitative analyses of test scores are misleading. How do we compare teachers when their classes are so different, when (other than GCSEs and SATs), all children are taking different assessments? Our partnership with our founding MATs presented an opportunity to overcome these problems. Children in our founding MATs typically take the same assessments, and rich anonymised data on their prior attainment and demographics (e.g. Pupil Premium or SEND status) meant that we could account for differences between classes. Theoretically, we could use anonymised MAT data to estimate teacher impact in a way that could be a game changer for our understanding of effective teaching.

We did not know if this would work! So, with our partners, we consulted teachers and school leaders in interviews and focus groups, before conducting several statistical tests to see if the anonymised data could be used for this purpose.

Here’s what we found:

1. As long as all data was anonymized, teachers and school leaders found it acceptable that we use their data to estimate teacher impact.
2. Although collating and matching assessment data required some careful preparation, extraction of anonymised assessment data from our partner MATs was feasible.
3. While a small number of assessments had room for improvement (some were too easy which meant too many children got the top score, and vice versa), the assessments were of sufficient quality to estimate teacher impact from the results.
4. After using multiple statistical models to ensure our results were not just random statistical ‘noise’, we are satisfied that we have a meaningful estimate of ‘teacher impact’.

The journey doesn't end here. The NIoT will now build a secure and open data infrastructure so other researchers can analyse anonymized data from our partner MATs, establish an Expert Working Group with teacher development experts to ensure our analyses are grounded in the lived experiences of the sector, and pilot cutting edge AI tools to identify the classroom practices of our most effective teachers.

Our ambition is to identify and share practical improvements to continuous professional development that can benefit all teachers and more importantly, their pupils.

For more information about PiloTED and how data might be used to enhance teacher development, you can read our summary and full research documents here.

About the author

Dr Raj Chande is a Senior Research Associate at the National Institute of Teaching, and teaches Mathematics and Economics at a secondary school in Hackney. Raj previously led the Schools and Early Years portfolio at the Cabinet Office's Behavioural Insights Team.