Mark Avery: If only our countryside were scruffier...

Wildlife loves scruffiness. Shaggy, untrimmed hedgerows bear more berries, rough, uncut grass hosts more insects and weedy corners attract more birds. In our intensively farmed countryside, wildlife struggles to survive, and a bit of scruffiness helps a lot.

Remarkably, the biggest source of wildlife-rich scruffiness in the European countryside in the past 15 years was not a carefully targeted, cleverly designed environmental scheme, but instead the introduction of an EU measure aimed at controlling agricultural production, called set-aside.

Remember the grain mountain and the wine lake? In 1988, set-aside was introduced to limit Europe's overproduction and the amount of money that was being wasted. The idea was simple – each year farmers would leave fallow five to 10 per cent of their cropped land. They were compensated for lost income, production was limited and the taxpayers' money was saved. Wildlife was not part of the thinking. Yet at the stroke of a bureaucrat's pen, nature was accidentally thrown a much-needed lifeline.

Set-aside created a network of thousands of scruffy fields across the country. Skylarks and lapwings can nest in them, rare arable plants are found in them, brown hares have their leverets there, insects buzz around and seed-eating birds such as linnets, corn buntings and yellowhammers benefit from the extra food. In France, the threatened little bustard, a bird of open plains, now depends on set-aside.

While wheat prices were low, set-aside was acceptable to farmers, but now grain prices have rocketed. Farmers get more than £130 per tonne for wheat compared with about £55/tonne in 2005. The increase is because grain demand has risen to supply biofuels and the increasing Chinese and Indian populations. With grain prices high, and predicted to go higher, farmers want to cash in, and set-aside is seen as a restriction on profits. Understandably, farmers lobbied hard for set-aside to go.

Policymakers were sympathetic, since the system of public payments to farmers has changed as well. In an era of subsidies, paying for land to be left unfarmed made sense, but now payments to farmers are unrelated to production (farmers get the same amount of taxpayers' money whether they produce much or little wheat), there seems scant justification in intervening to limit that farmer's economic choices.

We expected the 2008 round of agricultural reforms to abolish set-aside, but also to introduce measures replacing its environmental benefits into the future. Instead, EU Agriculture ministers, including our own Hilary Benn, rushed into setting set-aside at 0 per cent for 2008, with no discussion of the environmental consequences. Officials from the Department for Environment, Food and Rural Affairs (Defra) knew the environmental benefits of set-aside and we assume they briefed their minister. However, it seems that a whisper from the National Farmers' Union echoes so loudly through the Defra corridors that wildlife's cries for help are unheard. UK farmers are happy because their payments from the taxpayer are based on what they received under the old system in 2000-02, which included set-aside. So next year, they will still get their set-aside "compensation" amounting to £173m, but be able to grow wheat on that land – akin to being paid unemployment benefit and then being allowed to get a well-paid job.

Defra officials are scrabbling around, finally realising that they have an environmental problem on their hands. Mr Benn has some tough choices to make. We hope he will not turn his back on wildlife and do nothing. If he does, then it would put at risk Defra's own target, agreed with the Treasury, to stop the decline of farmland birds by 2020. The last time set-aside was reduced dramatically, but temporarily, the Government's measure of farmland bird numbers fell 5 per cent – yet when set-aside area increased again, so did farmland birds. Set-aside has been the safety net that has prevented further falls.

Mr Benn has two options if he wishes to replace set-aside's wildlife benefits: regulation or incentives. The regulatory approach would require farms to have an area of wildlife-rich fallow – much smaller than under set-aside, but managed specifically to replace the benefits which set-aside accidentally provided. Farmers would have to produce this habitat as a condition of receiving public payments for their whole farm area. This would put back an element of the regulation that removing set-aside has removed – but then those set-aside compensation payments are still to be paid next year, so this does not seem such a big ask.

The incentive option would pay for the benefits of set-aside as a special conservation measure – but the taxpayer and Treasury might balk at finding more money unless that £173m could be taken out of the subsidy pot and used to fund more conservation.

Neither option is getting a sympathetic reception from Defra, which claims there are procedural difficulties in implementing anything new. So Mr Benn will need to rattle some cages within his department if he is to do wildlife some good. It seems easier for policymakers to help wildlife by accident than by design.

The countryside may be tidier next year. But it will be a quieter place, too, with less birdsong and fewer buzzing insects. Just as wildlife loves countryside scruffiness, so should we.

The writer is the director of conservation at the Royal Society for the Protection of Birds