Can musical inspiration be found back in the USSR?

The apartment of the artist and singer Irena Doubrovskaja is about 20 minutes walk north-east of Alexanderplatz, the huge urban space that was more or less the dead centre of East Berlin during the Cold War, and which stands as one of the few remaining architectural remnants of the Soviet past. Doubrovskaja has lived here since escaping the chaos of post-Soviet Russia in the early Nineties, having studied fine art in Moscow during the last years of the USSR. Art is her main preoccupation. She paints frozen winter landscapes on world globes and makes art from Soviet detritus.

Doubrovskaja's apartment is firmly on the unrenovated side of the street – giant hallways, heavy, dark wood doors, antique plumbing and ceramic stoves. It was here that she gathered together ErsatzMusika, a band of émigré Russian musicians on Berlin's underground scene. They probe and drain the Soviet and Slavic in their shared histories, assembling songs that tell of how it was, and most importantly, excavating the human remains embedded in the Soviet system that defined their past as much as the engine of global capitalism defines their present.

When I arrive as dusk falls on a brilliant clear winter's day, more than half the band is already sat around a kitchen table piled with food. Grazing over the food and drink at the table there's the guitarist Leonid Soybelman, in his thirties, with tight-curled black hair; the percussionist Mikhail Zhukov, a famous orchestra player, white-haired and red-faced, and the young bassist Igor Vdovchenko.

Watch the ErsatzMusika video for 'Upon The Earth'

They explain how they cut their acclaimed debut album, Voice Letter, at the end of last year, recording in a tiny studio just up the street from Doubrovskaja's apartment, in a tiny room on the far side of a dark internal courtyard, ceiling and walls lined with egg boxes for sound insulation, the space stacked with bulky musical equipment from a pre-digital age as remote now as the Cold War.

The album takes its title from the musical postcards, or "voice letters" – 10in flexidiscs – that people would post across the USSR, containing spoken greetings or just a favourite song. Doubrovskaja digs one out of her collection. "They come from the 1960s and 1970s," she explains, "People would send them as presents for their friends." She turns the disc over, showing me the low-quality grooves cut into the plastic. "The song on this one is called 'Black Eyes', which was a gypsy hit. It was a very big song in Russia in the Seventies."

There's music on cassette, bikes in the hallway, a stack of vinyl at the end of the corridor, and, in the next room, amid half-painted globes and canvases turned to the wall, stands Doubrovskaja's upright piano. It's the piano, along with the curt chop of Soybelman's guitar, that provides the signature sound of Voice Letter, and it's an album that has captured imaginations worldwide. Its distinctive, spindly sound raises musical ghosts from the wreckage of the old USSR, blending poetry with folk-song and personal stories from the post-Soviet Russian diaspora.

"I started just recording them for friends," says Doubrovskaja, "to recreate what we used to have in the Soviet Union, the voice letters."

The album has the multi-layered depths of a collage, mixing found and favourite texts – anonymous Gulag poetry, a drinking song by the poet Julia Belomlinsaya, a picture of Leningrad in wartime by the Russian singer Leonid Polijaev – with musique concrète textures such as the mechanical, otherworldly sound of the music box on track one, which acts as the album's opening charm before the jagged-edged guitar chimes in alongside the sound of Doubrovskaja's wheezing, antique fairground organ.

The band's name comes from the ersatz coffee that people drank in the Soviet era, and at the same time nods toward their collective memories of the ersatz music promulgated by the authorities. The kids, of course, weren't listening. "All of us grew up on Western music – punk rock, rock music. There was folk music, but it wasn't really folk. Some orchestras would play folk songs. It was made very official." The authorities may have filled the air with strings, but it couldn't drown the sounds coming from the West, or from the USSR's own samizdat underground of Bardic and "criminal" singers, both of whom are core influences on Voice Letter.

"It was a cassette culture until 1989," confirms Soybelman. "A band made a record and put it on cassette and within a few weeks it would have crossed the Soviet Union. It was still underground and unofficial. There was no official outlet at all."

Later, Doubrovskaja pulls out a crate of some 200-plus tapes from the Seventies and Eighties, any one of which could have landed her in a labour camp if they had been found. These are criminal songs, a kind of Russian gangsta-rap usually featuring just voice and guitar, playing a distinctive choppy rhythm that's somewhere between a forced march and a ska beat. Political, personal and direct, it was this music the Soviet youth was tuning into, along with British and American rock. Doubrovskaja pulls out 30-year-old reel-to-reel tapes scrawled with familiar names like Slade and Deep Purple ("Slade were a huge hit in the Soviet Union"), alongside bands no one remembers anymore.

But the strongest outside influence on Voice Letter is the early Eighties British underground – the forbidden music they would have heard on samizdat cassettes as teenagers. There's raw snatches of punky rockabilly on "Tsaritsino Station" that's reminiscent of early Bad Seeds, alongside raw Ukrainian gypsy music in the form of "Eh Karina". The wonderful "Mushroom Hunter" borrows its descending piano riff and open sonic spaces from early Eighties Fall.

They have yet to play far outside of the clubs of Berlin but look set to travel a long way. "All of us have travelled a lot, and discovered freedom in very different ways. We don't have any system or scheme, just keep being yourself," says Doubrovskaja. "We don't have an explicit programme."

"English bands like the Kinks used folk music and our music is the same – we wanted the same kind of atmosphere, so though you don't understand the words you know what the song is about. I listened to English music without knowing what the words meant, but when I understood them I was amazed to discover that I already knew what it was about."

She pauses, looks down at her drawer full of criminal songs, the Slade reel-to-reel beside the box for a cassette, lampooning Stalin, that has been inserted into the cassette player. "From music you can really get the proper information."

'Voice Letter' is released on Asphalt Tango