Bill Wyman: 'Si si, I'm a rock star – but I'm a photographer too'

On "Si Si (Je suis un Rock Star)" Bill Wyman sings about picking up a Brazilian woman and taking her via Concorde to his south of France residence. But while spending the Seventies in Saint-Paul-de-Vence, France, the Rolling Stone wasn't just using his exotic homestead to his advantage in the bedroom. He was also hanging with the artist Marc Chagall and learning about art.

"I knew nothing about art until I went to France," Wyman tells me. "But I met all these bloody great artists who are like gods – Chagall, César, Arman – and, although I was in awe of them they treated me as an equal. They thought I was an artist of music."

Wyman, who tells me he has a store of 20,000 photographs, is known as the Stones' "librarian" for his obsessive diary-keeping – of which the photography is an extension. He has been taking pictures since the age of 11, when his uncle gave him a Box Brownie camera. But it was his friendship with Chagall that really ignited Wyman's interest in fine art.

"I really do look at things. The average person walks along looking at the ground," he says. "But I don't do anything special, I just take the bloody picture."

After first being championed by his friend the photographer Terry O'Neill, Wyman has had over a dozen photography exhibitions. He is in London at the Rook & Raven Gallery to promote the latest, a "rather special" one for him, which includes "re-workings" of his snaps (from landscapes to moody shots of Mick Jagger and Jerry Hall) by contemporary artists including Pam Glew and Wyman's friend Gerald Scarfe, the illustrator behind Pink Floyd's The Wall visuals.

During his time living in France Wyman was over at Chagall's house "all the time" and on one on occasion noticed a beautiful Picasso vase. "But that's a Picasso! How did you get this?" he cried. "I gave him one of mine, and Picasso gave me one of his. We swap," said Chagall. Wyman responded with: "But that's what I do with Eric Clapton! We swap new albums. I never thought real artists did things like that."

He was born William George Perks in 1936 later changing his name to Wyman by deed poll in tribute to a friend with whom he had served in the Royal Air Force. A bit older than the other Stones, he stepped in as their bassist in 1962 after Dick Taylor left, and was reportedly asked to join simply because he had his own amplifier. Wyman was the first of the Stones to break out on his own in 1993 after a 30-year stint in the band.

Since then he has had success with The Rhythm Kings, run his Sticky Fingers restaurants, become an accomplished archaeologist (even patenting his own brand of metal detector), written books… "My life is full," he says. "I work from morning to night just taking time out to go to the toilet, make a cup of tea and have my dinner."

At 76 his shaggy hair is fully grey and his bold eyebrows and angular face softened. Sitting in the gallery's chilly basement he moves closer to the small radiator and often cups his hands around a mug of tea for warmth. He is fresh from performing live with the Stones for the first time in 22 years at the band's 50th-birthday celebrations in December, an experience he describes as "short and sweet, I suppose you could say", having consisted of just two songs.

Talking of his "fantastic girls", three teenagers aged 18, 17 and 14, with his third wife Suzanne Accosta, his voice fills with pride. "[The 02 gig] was the first time they'd seen me play with the Stones, you know. They'd been to Stones shows before but never seen me on stage with them, so that was kind of nice. I did get some quite rousing applause when I went on."

Sadly Wyman is cut short by a rather overpowering PR who repeatedly vetoes any questions unrelated to the exhibition – which is tricky as the paintings are yet to be delivered.

Under the circumstances I refrain from asking Wyman about the Jimmy Savile scandal, but I'd have been keen to hear his view as a man who became notorious for his relationship with a minor.

His relationship with Mandy Smith began secretly when she was 13 and became sexual when she was 14, according to Smith. In an interview with The Observer in 2006 Wyman said: "I don't know [how it happened]. It's the magic of the moment, basically. It was 1984 when I met her. We got married in 1989. That was a disaster." They divorced in 1991 and Smith subsequently told the Daily Mail: "People aren't as protective of celebrities nowadays and if this had happened recently I think Bill would have ended up in prison."

It is hard to reconcile this, or indeed the Wyman reputed to have bedded a thousand women, with the likeable, domestically minded person in front of me. Is he happy? "I've never been happier," he says. "I'm a father and not a grandfather. At my age my three gorgeous daughters should be my grandchildren, shouldn't they? But they're my children. And I'm very fortunate since I can spend more time with them than I could have if I was younger because I would have been touring."

Bill Wyman-Reworked, Rook & Raven, London W1 (020 7323 0805) 21 February to 31 March