"High Stakes" Assessments Fail to Deliver

I felt very fortunate to recently hear a presentation from Margaret Wu, an internationally respected authority on educational assessment. If only the National Government had sought her advice before embarking on the ideological nonsense that formed the basis of their National Standards, then we may not be dealing with our current mess.

Margaret's quietly authoritative style and conclusions based on a distinguished career in educational research effectively exposed the myths that form the basis and justification for the National Standards.

Myth Number One: New Zealand's long tail of underachievement can be addressed principally by raising the quality of teaching.

Given that New Zealand is already ranked in the top five of education systems internationally it was always going to be difficult to significantly improve an already strong system. Margaret made the enormity of that claim even more ridiculous by presenting the determining factors for any child's academic attainment. When all factors are accounted for; family income, ethnicity or culture, past experiences etc - the influence of a teacher is only 10% of all determiners. With the current consultation on the government's Green Paper for Vulnerable Children, most people are beginning to realize where our attentions should really be aimed. To claim, as the government has, that failing children are the result of a failing education system and poor teaching is highly misleading and patently untrue.

Myth Number Two: National Standards will provide the Ministry with useful data to identify failing schools.

Margaret explained how any raw data provides little information on causes and any attempts to correlate data can be misleading. She gave the example that when there are increases in ice cream sales there are always similar increases in crime, the real relationship between the statistics is that warm weather contributes to the increases in both and there is no direct correlation. Margaret explained how the differences in achievement between schools are more likely to be related to income inequities between communities than school performance.

The fact that the majority of New Zealand schools have just over 100 pupils or less is problematic because the numbers are too small to draw useful conclusions on school performance. I recently taught in a rural school that suffered from the transient nature of the dairy workforce. On "Gypsy Day" the school could easily find 25% of the children would change and only around 50% of any class would have pupils who had been in the school for most of their education. To use achievement data in these schools to assess school performance ignores the small amount of influence a school may have on some children's learning.

The existing qualitative assessments provided by ERO  are already able to identify struggling schools better than National Standards will ever do. The National Educational Monitoring Project (NEMP) also provides high quality research that looks at the strengths and weaknesses of current teaching practice in each learning area in a more practical way than National Standards ever could.

Myth Number Three: The creation of league tables from the National Standards data will help parents choose the best school for their child and provide assurances regarding their school's performance.

Margaret was adamant that conclusions drawn from raw data, without qualitative support, have very little value. Parent communities will not be able draw any accurate conclusions from such comparisons to usefully guide their choice of a school because their ability to interpret the data will be limited.

Myth Number Four: National Standards will force schools to lift their performance, especially when the data will enable comparisons between schools and create competition.

Margaret called this scenario "high stakes assessment" and claimed it created data that was not reliable. Even with the best of intentions and professionalism there will be a temptation for teachers and schools to slant their results more positively than what may be the reality. If a teacher was asked to assess a child to determine what level of support they may need to assist their learning, then that assessment is more likely to be focussed on the child's needs. If you asked a teacher to assess a child's attainment so that the assessment will be used to determine their ability as a teacher and to help rate the school, the focus has shifted from meeting the child's needs to something completely different. The accuracy of teacher judgements would understandably vary greatly depending on other pressures and expectations being placed on individual teachers and each school.

Myth Number Five: National Standards will lift child attainment because of their "aspirational" settings that are above current norm-referenced assessments. 

Overseas research has found that National Standards systems did produce a short term lift in achievement, but Margaret noted that such leaps in achievement, especially in "value added" models are unsustainable. Children do not continue to make huge leaps in their learning year after year. Also, unless the other factors contributing to the tail of under achievement are addressed, any broad improvements in the education system may lift all attainment slightly but the spread of achievement will remain the same.

Myth Number Six: The publishing of each school's National Standards data will just highlight the results of demographic groups and the privacy of individual students is assured.

Margaret did not directly comment on this except to refer to the difficulties of assessing the performance of small schools where pupil numbers are not enough to allow for any accuracy in conclusions. The concern from schools and teachers from any public release of the data is that even though children aren't identified by name it will still be easy for school communities to identify the results of individual children. Now that schools have been told to report on the achievement of all children, if they have only one child with an established disability their scores could be easily identified from amongst all the others. Similarly if the results of Maori or Pasifika children are highlighted and there are only a few of these children in a school it would be relatively easy to attribute individual scores to the child. This is a highly unethical situation.

I found Margret's presentation and her answers to our questions hugely valuable, many of her conclusions were not new, but the fact she could express them so succinctly and support them with valid research was very useful. It is hugely frustrating that such useful knowledge and our own academics are still being ignored by the National Government as it continues to implement this highly flawed assessment system.

A recent meeting of NZEI Te Riu Roa with the new CEO of our Education Ministry made it very clear what level of engagement the profession will have regarding any changes over the next three years. Under the new regime teacher organisations only exist to represent teachers on industrial matters and will not be consulted regarding professional issues. The government has already indicated that there are too many people in the Ministry with links to NZEI (through their teaching backgrounds) and their influence needs to be reduced. I was recently talking to someone who had just left the Ministry and she described an obvious culture change as those with bureaucratic and administration backgrounds were replacing those who had always worked in education.

I can see another three years under a National Government being a very challenging one for our quality public education system, or even maintaining the quality we currently have. One of the biggest concerns for most of us who work in education is the narrowing of the curriculum due the Standard's unhealthy focus on literacy and numeracy.