Marks and sparks

Shock! Horror! Scandal! The pass mark for this year's national tests for 11-year-olds in English has been lowered by four. It looks very fishy. First we have David Blunkett, the Education Secretary, saying he will resign if 11-year-olds don't reach targets in English by 2002, and then we get a newspaper report that he has resorted to making the tests easier.

What a good tale. In no time at all it has gone round the world and media in Australia, New Zealand and America are busy casting doubt on the reliability of our tests. Back home, even the panellists on BBC's Have I Got News for You are joking about the cunning Mr Blunkett "fixing" tests. Finally, cunning Mr Blunkett holds up his hands and announces an inquiry into the whole murky affair in order to clear his name and put the tests in the clear, too.

But hang on a minute. Even a cursory examination of the procedures for agreeing the pass-marks for tests shows that Mr Blunkett could not possibly have had anything to do with it. Such questions are decided by a panel of experts without any reference to the secretary of state, or his officials. Indeed, insiders tell us, Mr Blunkett was as surprised as anyone else to discover that the pass-mark had been lowered.

Even if he had known, he should not have had the slightest cause for concern. Anyone who understands how tests and exams are marked from primary school to university knows what the inquiry team will find. It will discover what most fifth- and sixth-formers already know - that the pass-mark for this year's English national tests has been decided in exactly the same way as the marks for GCSE and A-level - and for all the other national tests.

What is more, these marks are just as likely to be raised as lowered. There is no scandal. There is no fiddle. Examiners have simply made allowances for the fact that the inexact science of examining means that the difficulty of papers varies from year to year. Far from lowering standards, the adjustment of pass-marks is the only way of ensuring they are maintained. George Turnbull, of the Assessment and Qualifications Alliance, one of three national exam boards, says marks are changed at GCSE and A-level so that standards for each grade remain the same. "There is nothing sinister about the changing the pass-mark. Marks may fluctuate by quite a number if a paper is harder or easier."

Examining is a tricky business. Even in the days when most children took the 11-plus, examiners found it difficult to ensure that questions were equally challenging each year. In some years, for example, 100 children would reach the previous year's pass-mark and, if there were 120 places in grammar schools, the mark would be lowered to ensure that all the places were filled. Equally, if 140 children passed, the mark would be raised due to the restricted number of places available.

But until the early Eighties, public examinations at 16 and 18 plus (O- and A-level) were a comparatively straight- forward affair. The same percentage of candidates was allocated to each grade each year, regardless of individuals' performances. That meant that even if schools and pupils worked harder and more reached a higher standard, their achievements could not be recognised. The system was fundamentally elitist. Progress was impossible, and neither teachers nor pupils had much incentive to improve. Today's way of deciding the difference between an A and a B at GCSE and A-level, for example, or a Level 4 and a Level 3 in the national tests for 11-year-olds, is much fairer, but more complex. Various checks and balances are built into the system, including statistics on how candidates performed in previous years.

One of the main ways in which examiners ensure that the standard of each grade remains unchanged is by making judgements about the difficulty of the paper. For the tests, an expert panel of test developers, chief markers, researchers and Qualifications and Curriculum Authority officials, who advise the Government on exams, decides in February to March where the thresholds should be set. After the tests are taken in May, a further meeting in June takes the final decision on thresholds. It looks at completed exam papers where the marks achieved are on the borderline between two grades.

Exam boards operate in a similar way. A meeting of experienced examiners looks at this year's borderline papers and compares them with those of previous years to determine where the threshold for the grades should fall. They discuss whether the paper is easier or more difficult than those of previous years, and also make allowances for questions that have been misinterpreted as they were not clear.

So how do testers and examiners determine a paper's difficulty? English poses particular problems. The 11-year-olds' English test this year was thought to be slightly trickier than last year's as part of the reading comprehension was a poem, a form of literature which experts felt was less familiar to most children than straightforward prose. It was also allusive. The subject was spiders, but the title was Spinners, which some children associated with yarn and wheels, rather than spiders.

Sometimes it becomes clear that a substantial number of children have misinterpreted a question. A GCSE comprehension piece about sending an elderly relative to a home, for example, proved incomprehensible to some candidates, particularly members of the Asian community.

The process of adjustment may be complicated. Anne Barnes, an experienced GCSE and A-level English examiner, says: "You may have an exam where the first section is about reading, and you find the candidates all get very high marks. But they may then get low marks on the writing, and so there is an imbalance between the two."

An additional check is built into the procedure for ensuring that the standard of national tests is maintained. A few weeks after one year's national tests are completed, draft tests for the following year are taken by several thousand children so that officials can assess whether they will prove more or less difficult than in previous years. The result is a pass-mark that moves both up and down. The last four years have seen, for example, an increase of four in the pass-mark for maths at 11, a fall of four and an increase of one in the mark for science. There have been even bigger changes in the pass-marks at 14.

And the figures show that the allegation that this is all a New Labour plot to help Mr Blunkett towards his targets is just silly. Some of the biggest downward adjustments to pass-marks were made under the Tory government. For the English tests for 14-year-olds taken in May 1997, the pass-mark plunged from 40 to 27. (This year it is back up to 36). For those for 11-year-olds in the same year, it went down from 57 to 52.

What would happen if adjustments were not made? Anne Barnes says: "This is the only way you can possibly maintain standards. If it didn't happen, you'd get completely erratic results. Schools would complain they had the same sort of ability range among their candidates as last year, but the marks were incomprehensible."

Universities go through a very similar process for degrees. An external examiner checks the paper to decide how it compares with previous years.

How can we be sure standards are maintained over time? All checks and balances help ensure standards are maintained. But an extra check is being made on the standards of English and maths tests for 11-year- olds. Children in Northern Ireland, where the 11-plus still exists and mainland tests aren't used, are taking the English and maths tests for 11-year-olds used in England from 1996 to 1999 to check if the level thresholds set for this year's tests are equivalent to those of 1996. The Government will get these results in the autumn.

Exam officials take immense pains over the adjustments they make to be fair to pupils and to maintain standards. The process will never be exact: exam results will never be totally reliable and the "standards are falling" brigade will always believe there is a conspiracy to dumb down.

And what is the alternative? A return to the days when only a fixed number of mainly middle-class children passed exams? One of this Government's most admirable goals is to end the notion that only a minority of pupils can reach an acceptable standard either at 11 or 18. It will be sad if ministers let themselves be blown off course.

THE UPS AND DOWNS OF PASS MARKS

The following are examples of pass marks at Level 5

of the national curriculum tests for 14-year-olds

1995 1996 1997 1998 1999

English 41 40 27 40 36

*Maths 32 30 29 37 36

*Science 54 43 48 51 39

Pupils are expected to achieve between Level 5 and 6 at age 14

*Papers are set for differing abilities. These results are from the test papers sat by the majority of 14 year-olds