The girl in the painting

There sits my grandmother, Olive Blake, as a child at a fancy dress party in the Bahamas in 1885, dressed resplendently as an Arabian princess with scarlet head-dress and broad cummerbund over a yellow dress. A fan, which looks as if it is made from peacock feathers, dangles from her left hand and beside her, dressed as Arabian princes in flowing white trousers, are her younger brothers Maurice and Arthur. No wonder their parents Sir Henry and Lady Edith Blake were pleased by the watercolour of their three children painted by the American artist Winslow Homer.

The picture, for all its troubled recent history – detailed last Sunday evening as part of a major BBC series, Fake or Fortune – has an appealing freshness and spontaneity about it and Olive and her brothers look attractive, without being overly self-conscious of their exotic costumes. There is a certain formality about their expressions and posture, as if they are conscious that their father, Sir Henry, is the governor of the Bahamas. I wonder what Olive or her parents would have made of the controversy now surrounding the re-appearance and contested ownership of this charming painting, well over a century after it was produced.

Did she ever realise that Homer, already well-established when he visited the Bahamas, had gone on to become one of America's iconic painters who enjoyed immense popularity and some of whose paintings sell today for millions of dollars? I only have a hazy memory of Olive, who died in 1953 when I was aged three, as being a formidable looking woman of whom I was somewhat frightened.

On the other hand, at that age most adults, aside from my parents and nanny, appeared to me to be intimidating and possibly hostile forces.

I remember when I was about three – it must have been shortly before she died – my grandmother took me and my first cousin Shirley in her car to bathe at a beach surrounded by rocks and cliffs called Goat Island near the town of Youghal on the southern Irish coast. Shirley and I both ran fully clothed into sea and were brought back shivering and in disgrace, wrapped in newspapers to Myrtle Grove, a Tudor house behind the medieval walls in Youghal once owned by Sir Walter Raleigh, where she lived.

Sir Henry and Lady Edith had gone to live in Myrtle Grove after he retired as a colonial governor in 1907 and they are buried in the garden. It was a few miles from here, a century after it was painted, that Winslow Homer's painting was picked up next to a rubbish dump beside Youghal bay by a fisherman called Tony Varney. It was found with some other papers, one of which established a link to Sir Henry's time as governor of the Bahamas. The finding of the watercolour was in the 1980s and it then disappeared into the attic of Mr Varney's daughter, Selina, for 20 years, until it was identified on The Antiques Roadshow as being worth £30,000, a figure which, according to experts in New York, is a serious underestimate. They suggest that the painting, which was being auctioned by Sotheby's, might sell for $250,000, but the sale was stopped at the last moment after objections by descendents of Sir Henry and Edith Blake.

Olive Blake, the central figure in the watercolour, would probably have been taken aback by what has happened.

My mother, Patricia Cockburn, the youngest of Olive's six children, described her mother as strong-willed, intelligent, interested in the world around her, but highly conventional. As the daughter of a colonial governor, Olive developed a taste for travel and was contemptuous of personal discomfort. She married Jack Arbuthnot, an officer in the Scots Guards and ADC to her father, who was then in his last year as Governor of Hong Kong in 1903. The wedding was a regal affair, with the men in dazzling white uniforms – and her new husband, who liked his creature comforts, had expected the honeymoon would be luxurious.

Instead, Olive insisted they should travel to a remote Buddhist monastery in Japan of which she had read. When they arrived, he discovered they would have to sleep on the floor on straw mats.

The monastery was also overrun by rats. Olive later gleefully told my mother: "You know, Daddy didn't like it a bit. He had to stay awake all night shooting at them with a revolver."

Jack Arbuthnot, who for many years supplemented the inadequate pay of an army officer with part time journalism, was the originator of the "Beachcomber" column for the Daily Express. He was also a good amateur painter and sculptor, but quirky and unpredictable. He fought in the First World War and made an original and, to me, very moving relic of his time on the Western Front. On Christmas Day 1914 he collected broken medieval glass from the cathedral at Ypres and reworked the fragments of brown, red, green and clear glass, into a metal cross.

Sir Henry and Lady Edith Blake would probably have been less affected by the strange history of Homer's picture of their children – something they apparently commissioned on the spur of the moment – because their own lives had been full of high drama.

Staring out from their official portraits in the newspapers of the day in the Bahamas, they look almost like caricatures of coldly remote British rulers at the zenith of the empire. But, in reality, their marriage was the result of an elopement, furiously opposed by Edith's wealthy Anglo-Irish family, which had disinherited her. On the death of her mother, who had inherited the family estates, Edith and Henry had tried to seize back a house – which she felt was part of her inheritance – at pistol point and were brought before a magistrates' court.

Edith Blake came from a rich, but dysfunctional family with large estates in Tipperary and Waterford, including extensive copper mines along the coast.

She was the elder of two daughters of Catherine Isabella and Ralph Bernal Osborne, who had come to loathe each other shortly after their marriage in 1844. He was a liberally minded Whig MP, who was originally called Ralph Bernal and appears to have married her largely for her money and resented having to change his name to hers.

An early dispute came when he tried vainly to have all the sheep on her estates in Ireland re-branded with his initials instead of hers.

Relations did not improve over the next 30 years. "The most violent scenes used frequently to take place between my parents," Edith later wrote. "My sister [Grace] and I often stood holding each other's hands in the corner, very much frightened. I hated my father and looked upon my mother as a suffering angel."

Frightened or not, Edith was not an inhibited Victorian girl. She painted extremely well and wrote an excellent book on travelling in southern Europe.

At one moment she complains vigorously about a hotel in Istanbul where she was staying and which had told her there was a public baths next door. In fact, it was some way off and, she notes irritably, that she had to walk to it through the streets of the city wearing only her dressing gown.

Hostilities between her mother and father persisted as Edith grew up. Things were made only slightly more bearable by the fact that he lived mostly in England, while Catherine Isabella stayed at Newtown Anner outside Clonmel in county Tipperary. When he did visit the house she would greet him by saying: "I trust you are well, Mr Osborne, and how did you leave your mistresses?"

When Edith was 16 or 17 her mother wrote a novel called False Positions, published anonymously, which was a thinly-disguised attack on her husband.

Formal separation between Bernal and Catherine Isabella was often mooted but never happened, possibly because relations were too venomous to achieve even this modus vivendi.

Aside from these infuriated rows, the Osbornes were highly educated, painted and drew well and had an early interest in photography. Often the women of the family posed in fancy dress as Italian or Balkan peasants.

Catherine Isabella, presumably ruing her own experience, saw all men who wooed her daughters as potential fortune hunters. Curiously this was almost the only subject on which she and her husband sometimes agreed. When Edith said she intended to marry a good looking and recently widowed police officer called Henry Blake who commanded the Royal Irish Constabulary in the local market town of Clonmel, they adamantly opposed the match.

Briefly united, her parents said he was an adventurer whom they alleged had once been a draper's assistant in Limerick. When they did marry in 1874, Edith was promptly disinherited and, in true Victorian fashion, her parents forbade her name ever to be mentioned in their house again. The newly-married Blakes were poor, compared to Edith's previous palatial standard of living. Henry resigned from the police and became a Resident Magistrate, a powerful post with civil and military authority, in central Ireland, just as the war between landlord and tenant was at its height. He was much hated for overseeing evictions, arrests and trials and an open grave was dug outside the Blakes' house to underline local hostility. Their doors were barred.

A visiting local journalist said that Edith acted as a sort of bodyguard to her husband, adding admiringly "she is always armed, a dead shot with a pistol and practises every day".

Edith found other uses for her gun. The bitterness between her and her parents remained deep. When her mother died on 21 June 1880, the Blakes took back one of the family houses. But while they were attending her mother's funeral they found that the agent for the estate had taken advantage of their absence to install his own caretaker. On their return to the house, Henry tried physically to throw the man through the door, while Edith drew a pistol, according to a later court report and shouted: "Look at this – if you don't go out I will put what's here through you." The Blakes never denied that she had drawn a gun and Henry's lame excuse was that she was could not have shot the caretaker because he was standing in the way and she could not get a clean shot.

The culmination of the family row came the next day in court when Henry was accused of using "force and violence" and Edith of threatening to shoot the caretaker with her gun. The Blakes saw what had happened in a different light. Grace, Edith's younger sister, had married the Duke of St Albans and they stood to inherit the Osborne estates.

In court Henry, still enraged, somewhat illogically blamed the whole affair on the Duke, saying he wanted to get him sacked as a resident magistrate. He said: "The only one who has an interest in this matter is my noble brother-in-law, the Duke of St Albans, and if he has anything to do with it, I tell him it is an ignoble and discreditable thing for him to bring Mrs Blake here."

Though should have been an open and shut case, sympathetic local magistrates decided against putting him on trial and the crown did not appeal.

Possibly Blake got off because the beleaguered government in Ireland did not want to lose one of their more effective lieutenants. He also appears to have dropped his and Edith's claim to the Osborne estates. As well as being a magistrate he was a prolific and eloquent if anonymous journalist with a stark view of Irish politics.

He wrote in 1880 that "there are two Irelands, more clearly defined in religion, feeling and interests than were the Northern and Southern States of America in 1864".

The New York Times wrote that after a few years as Special Magistrate, with wide authority, that "Blake had made himself so widely hated by the people that he had to be removed from Ireland". Even so, the pay-off was munificent, probably because the government was in his debt because of active and dangerous service during the land wars, but also because he was now backed by the Duke of St Albans. He was given a knighthood and, after an attempt was balked by the Irish in Australia to make him governor of Queensland, he became governor of the Bahamas in 1884. It was here, months later, that he held the fancy dress party which his children attended in oriental finery and Winslow Homer painted the picture, the ownership of which has created so much controversy.

The Broken Boy, by Patrick Cockburn, is published by Jonathan Cape