James Ivory in Conversation, by Robert Emmet Long

Yet I grew up with Merchant Ivory's delicately subversive vision of post-Raj India, taken to see my aunt - one of their stalwart actors, Madhur Jaffrey - in films such as Shakespeare Wallah and Autobiography of a Princess. Later, I found in them an antidote to the dubious nostalgia of the "Raj revival" in 1980s Britain. Even Heat and Dust, based on Jhabvala's Booker-winning novel, was subtly at odds with the vogue it led.

These conversations with Ivory, completed before Merchant's death, group his features over 44 years by setting. As he says of filming in Paris, "I can only work, as a foreigner... from the outside in - as I had to in England, and as I did in India". Born in Oregon, to an Irish Catholic father and Louisiana-born mother, he describes his growing love for India and friendship with Satyajit Ray, and the balancing act of staying independent and low-budget despite links to Hollywood. Even before A Room with a View brought big studio offers ("They thought we had some secret"), Columbia had bought their first feature with "frozen" rupees, which could not be repatriated and funded India-set films.

Although Ivory embraced period "authenticity", he dislikes the adjectives that stick to his work - decorous, languorous, pallid and stately - though drily conceding that "none of my films are exactly noted for their speed". He views Howards End and Remains of the Day as comedies, his outsider's eye recreating their gorgeous surfaces while keenly discerning the structures beneath. Erupting, he says, "But we don't really regard the tradition of princely India as being sublime, do we? Under the veneer of glamour lay a bloodthirsty history of a feudal state."

Charles Wilcox in Howards End is a "sort of middle-class English snob, of a certain dangerous kind that Forster always had it in for". As for protests over Jefferson in Paris before DNA tests proved the president had black descendants, Ivory is clear that people who thought it "impossible for Jefferson to have had a slave mistress because he was too fine a person ... are racists".

Ivory's films may benefit from an outsider's eye in another sense. On Maurice, Forster's long-suppressed story of homosexual love, he says the hero "felt 'different' from all the Charles Wilcoxes he knew ... But he'd been born with all the same snobberies."

The director is silent about his own homosexuality, though his films often question constructions of masculinity. "They train themselves," he says of American men, "or should I say we train ourselves not to show what we feel." Through tantalising gaps, the young Ivory emerges as "too pretty for a boy", "lanky and skinny, with a reputation for being a snob and a know-it-all", who was often beaten up. His father enrolled him at the gym and taught him to shoot. Observing him on Sundays at altar-boy duties, he encouraged his son's view of himself as a performer, coaching him. "These must have been a little like the notes a director gives his actors."

As a conversational fencing master, Ivory is elegantly poised, and prone to lethal thrusts: the Laura Ashley joke "will be remembered long after Parker's own films are forgotten". He can be gratifyingly indiscreet, and confesses to cherishing the handkerchief with which he wiped spilt whisky from Catherine Deneuve's knee. But the guard on his own privacy never drops. "I always have the final cut."