Westminster's biggest hate figure?

The Speaker of the House has given us a complex display of virtue, vice and human flaws – along with an unconventional wife with a colourful past and an intensely political present.

In the corridors this week John Bercow got into a brawl with Mark Pritchard, deputy chairman of the 1922 Committee. Bercow demanded that Pritchard show him more obvious deference: "You will obey the courtesies of the House!" And Pritchard finished the very public exchange with: "You're not the fucking Royal Family, Mr Speaker!"

That is bruising stuff. The Speaker is the fourth most senior non-royal Briton, but the dignity that should accompany the position remains elusive for Bercow, whose flaws may yet undo him.

Many of us gathered in the Gallery – the "snobs and bullies" especially – laugh at the Dickensian parody of his speech, his nakedly self-interested canvassing, his need to speak so much from the chair ("Any more points of order? None? Not a little one?").

The Pritchard affair this week might not have mattered so much in isolation, but before Christmas Bercow made an enemy of the Government's Chief Whip. He chastised Patrick McLoughlin extensively, publicly, and – to Tory eyes – gratuitously. The expression on Bercow's face is available to all on YouTube and by golly it repays study. The anger is freaked with dislike. McLoughlin, who is built like a strike-breaking Derby miner, is said to be angrier now than he was at the time. He is not a man to be taken lightly.

Friends point out that Bercow is a reforming Speaker in an institution that resists reform with a medieval intensity. He has made more time for urgent questions, and he promotes the rights of backbenchers. He is extremely popular with Labour, the Doorkeepers and the Youth Parliament – their admiration rises like incense from the floor of the House. But recently, his vices have caught up with him.

It was clear some years ago that Bercow was going for the Speakership and was going to win it with Labour votes. To that end, he began a systematic campaign of mocking and belittling his Tory colleagues and sucking up to Labour.

The Speaker is supposed to be above politics. Yet Bercow remains intensely political and he clearly hates Tories, as his dealings with them on the floor of the House make clear. When Labour misbehave he calls out something like: "I don't know what the hon gentleman had for breakfast but I must try it myself." When Tories misbehave it is more like: "This barracking is repulsive. You will be silent or leave the chamber, I don't care which."

A pro-Bercow whip in the Government said he was surprised at the "unreasonable level of anger" among the new intake of MPs.

It is less likely that there will be a vote of no confidence in the Speaker (MPs shrink from that sort of confrontation). But he might start to lose control of the House as passions rise, and that would have a serious effect on his future. The first virtue in the House is order.

Bercow is an extremely accomplished operator. In committee he will remember the special interest of every member round the table. In the House he will say to some ancient Tory: "The hon Member who came to the House 27 years ago on Tuesday..." – and how they love that recognition. He has done scheduling favours for this right-winger who ought to be an enemy, and constituency favours for that prominent left-winger. He has friends in surprising places.

But then there's his rage. White, privileged, Oxbridge-educated grandees – and there are plenty of them in the Palace of Westminster – set Bercow off. "I wouldn't talk to a dog like John talks to X (a clerk)," a colleague says. So there are dramas brewing backstage. And that's the continuing story.

Simon Carr's biography of John Bercow is out this spring