A version of this essay was first published in the British Journalism Review.

For a journalist the Court of Protection is like no other court. Whether you're sitting in on the trial of the year at the Old Bailey, covering a routine case at a local court, trying to make out what the Master of Rolls really means in his forty page judgement or even watching an intensely personal custody battle in the family courts, there's a general acceptance by all parties that reporters are a necessary and indeed vital part of a transparent judicial process.

Not so the Court of Protection. Lawyers often run a mile when you introduce yourself to them while some clerks look dumbfounded when you show your press card. A reporter at the Court of Protection is viewed with deep suspicion – an amoral muck raker who wants to delve into the private lives of Britain's most vulnerable people to fill column inches.

Transparent

Nothing could be further from the truth. Those of us who cover the court are more than aware that people who are unfortunate enough to need its protection are deeply vulnerable and they deserve their privacy. But wherever possible justice must be transparent. And for many years the Court of Protection came up short in this respect.

Rushed through in the legislative wash up before the 2005 General Election, the court has been controversial from its inception. Complaints soon flooded in: People said it was cumbersome, it was expensive, it was insensitive to the loved ones of those it was trying to protect. But if they spoke out publicly they faced being in contempt of court.

Moreover some of the decision judges were being asked to make were extraordinary, both in medical and legal terms and often involved requests to breach some of our most fundamental human rights.

Journalists who were interested in the court (and even now it is still just a handful) wanted to know whether there was substance to these allegations. But they found themselves head butting a brick wall. As most proceedings were held in private – and hence all the decisions the court made and how they reached them - there was simply no way of telling fact from fiction.

The Independent's campaign to report on the workings of the court – which is as much a pioneering legal battle as it is a journalistic one – helped changed that. And we've come a long way since we were first allowed to report a case in the spring of 2009 on the blind musical maestro Derek Paravicini.

In the past 18 months the public has been told of a string of vital cases including – and this list is by no means exhaustive - a decision not to allow a woman in a minimally conscious coma to have her nutrition withdrawn; an elderly couple's fight to go on holiday after their local authority refused them permission; a mother's bid (eventually withdrawn) to have a daughter with severe learning difficulties sterilised because she kept getting pregnant; and numerous disputes where children have been removed from their parents – rightly or wrongly – by local authorities. None of these could have been made public before.

Details have also emerged for the first time of how the Court of Protection intervenes to prevent serious wrongdoing. The famous case of Steven Neary, a young autistic man who was illegally removed from his father by Hillingdon Council, made headlines across Fleet Street. Sadly that case was not a one off. We also now know an 18-year-old man with autism and severe learning difficulties was unlawfully deprived of his liberty by Wigan Council when he was placed in a padded cell 192 times in a single month. We also know that employees of a care home in Cheshire were sacked after they doctored the records of a difficult patient in their care.

We've also been able to gain an insight in some of the desperately difficult calls that local authorities have to make and judges have to arbitrate on. Should someone with intense mental health problems have their leg forcibly amputated to save their life from a potentially fatal infection? Does a gay man with severe learning difficulties have the capacity to consent to safe anal sex? How long should a mother who has stopped her son's medication be banned from seeing him?

They are intensely difficult decisions with huge ramifications which is exactly why the press should be present. But there's still a long way to go. At present the onus is still on the news organisations to show “good reason” as to why they should be let in to and report a case. This can often be a lengthy, time consuming and expensive process. It is something that even the nationals, with their ever restricted legal and editorial budgets, struggle to do - let alone local newspapers who are exactly the kind of organisations that should be covering the day to day workings of the Court of Protection in their patch.

Sheltered

Every now and then we come across a hearing where all the parties involved are happy – or at least begrudgingly accept – that the press should be present and there is no need to go to court. But these are still a rarity. The problem is that most of these “no contest” cases have such a compelling public interest argument – such as a serious medical procedure or clear prima facie evidence of wrongdoing – that the likelihood is the judge would rule that the hearing should be made public anyway.

Most of the time we have to fight it out before the hearing begins. Usually the argument used to deny the press access is that having journalists present and reporting would not be in the best interests of the person who seeks the court's protection. Generally reporters have no desire to identify the vulnerable person unless their case is already in the public domain. But there is a strong public interest argument to name the local bodies who are in charge of that person. People have a right to know what decisions are being made by officials in their area.

Yet too often it feels like local authorities and primary care trusts are using this argument to shelter their own – often controversial – decisions from public scrutiny. I've lost track of how many times I've heard lawyers for a local authority argue that identifying them might lead to the identification of “P” - the person in need of protection. Often there is little evidence to back this claim up other than a suspicion of jigsaw identification yet the courts almost always agree to a ban. The conflict of interest is clear. These local authorities would much rather do what they do without having to defend it publicly so they invoke the need to protect P when really they want to protect themselves.

It's an oddity when you look at the wider context. In the criminal courts and family courts, geographic areas and public bodies are routinely named while underage or vulnerable people coming before the courts are kept anonymous. It works there so why not the Court of Protection?

The attitude I often come up against is that we reporters don't really understand how complicated the court is and the risk is too great to let us in. I was at a drinks reception for lawyers recently and came across a senior employee of the Official Solicitor – the government body which represents “P” - who recoiled when I said it was a good thing that more cases were being reported. She not only felt their job was being made harder by all this pesky press intrusion but that we didn't understand how important it was to keep proceedings private.

That view is as patronising as it is wrong. Just look at the way reporting has evolved when it comes to sensitively detailing sexual abuse proceedings or trials involving under age witnesses. We're adept at filing fair and accurate reports of each day in court which keep the identity of the victims secret whist casting a needed spotlight on the public bodies involved in the case.

Another argument I have heard is that the courts are publishing more and more judgements which the public can read. That’s true to an extent. The legal online database Bailii does now have a small number of Court of Protection judgements – as do a handful of mental health blogs. However these websites are incredibly specialised and you need to search for them. The idea that the average member of the public would even know where to begin looking is fanciful.

Even when we're allowed in to a case there's no guarantee we'll get to report it. The second Court of Protection case we successfully gained access to involved a mother who had been forcibly removed from her son because the local authority decided it was in his best interests to do so. The case was complicated but essentially the local authority had fallen out with the mother because she had particularly unorthodox views on medicine – views which they believed were detrimental to her son's health. The mother, meanwhile, argued she was simply a devoted carer and that her son repeatedly expressed a desire to live with her. It was a heart breaking case involving the forcible separation of a mother from her son.

Grounds for hope

We sought access to the case and were fought against by the combined might of the Official Solicitor, the Local Authority and the Primary Care Trust. It went to the High Court where we won and we published a report in The Independent. Our opponents appealed that decision and it went all the way to the Court of Appeal where three of the most senior judges in the land upheld the initial ruling. It was a landmark judgement because – unlike with the Derek Paravicini case – it paved the way for the press to access a hearing where the identity of P wasn't previously public .

A few months later I went to attend a follow up hearing which had been scheduled to update the court on the boy's progress. We would also find out whether the mother should be kept separate her son for another lengthy period. Given that the public was already aware of the case it made sense that we would want to update our readers on any decisions the court made. Indeed the public had a right to know.

The case itself had returned to a district court (for legal reasons I can't say where) and it was heard over two days. At the end of the hearing the Official Solicitor, the Local Authority and the Primary Care Trust made another submission to the judge calling on him to ban any reporting on what had been heard. It was a classic legal ambush. Without representation I tried my best to argue that the case was already in the public domain in anonymised form and it made no sense to close us down. But I doubt I was much of a match against three trained barristers and the judge sided against us.

To this day although I know the latest – and have permission to attend all future hearings in this case - I can't report what has happened to the young man. I'd love to tell you whether he was reunited with his mother, or whether the local authority was right all along and he's thrived without her. But I can't.

But it's not all doom and gloom. There is growing recognition among the small number of legal experts who specialise in the Court of Protection that allowing the press into proceedings has not resulted in the privacy Armageddon that some predicted.

Lawyers who represent family members and carers that are involved in disputes with local authorities are often quite keen to people in the press seats. But even those who represent local authorities are starting to recognise it can be in their interest to open up to public scrutiny - especially if the court vindicates their clients over a difficult decision they made.

Most importantly a string of senior Court of Protection judges – through the judgements, comments in court, interviews and after dinner speeches – have all indicated that they'd like to see more anonymised cases heard in public. We now have to make sure all those involved in the court heed their call.