White Rabbits: Pulled out of the hat - what the next big thing did next

"I'm getting you guys some beers," says White Rabbits' tour manager. "You all look far too sober." It might be 1pm on Thursday but we're at South by Southwest (SXSW) music festival, a place where you're apparently urged to guzzle margaritas for breakfast. The restaurant we were due to conduct the interview in is unexpectedly closed, so we gather on a bench outside a fast-food drive-thru next to a busy road. Between chain-smoking cigarettes, three out of five members of the band – vocalist and pianist Stephen Patterson, guitarist Alex Even and drummer Matthew Clark – tell me why SXSW is so important to the band.

"We met the guy who owns our label here after he caught one of our shows," recalls Clark. "It's the classic story you hear here: you play a ton of shows, then somebody sees it and decides they care." Somewhat unusually for a rock band, they are being much too modest. When they first played SXSW in 2008, they were the talk of the festival and came away the industry's most exciting new prospect. And it's not surprising. While their records are certainly captivating, they are a band you really want to catch live and where they tend to impress the most, as anyone who has ever seen one of their shows will tell you. Buoyed by a second drummer, Jamie Levenson, and another vocalist and guitarist, Gregory Roberts, they throw themselves around the stage, swapping instruments, sharing vocal duties and generally giving it their all before leaving the stage sweat-soaked and drained.

The band is here to show off their third album, Milk Famous, released last month. For those surprised at the new direction the Brooklyn band have taken, they offer a simple explanation: they merely had more time to work on it.

"We spent over a year writing the material and it was a very different process to writing for our previous records," says Patterson. "With this one we were able to take our time and observe things and wait until we were feeling inspired to write a song. On top of that we experimented with a bunch of different arrangements and ways of approaching the songs. A lot of the songs have three or four different versions before we decided on them, which we'd never had the opportunity to do before."

Like their other offerings, 2007's Fort Nightly and 2009's It's Frightening, the acclaimed band continue to specialise in experimental indie-rock but Milk Famous is even more textured, more eclectic, filled with unusual sounds and playful directions that mean the listener never quite knows where a song is heading. While you could accuse them of being deliberately difficult, they want their music to challenge their fans.

"We get bored really easily so we tried new things and we incorporated new sounds," continues Patterson. "I was never worried about alienating fans because it was coming from us and if they liked our music, they would like this. You can't think too much about what people might expect because that's a stifling way to approach things. I wanted any one of the songs to be able to be a single. One of my favourite records is Jesus of Cool by Nick Lowe, which sounds like a collection of singles that work as a group. Every song has a different feel to it."

It is appropriate we meet in Austin as the band, all in their late twenties and early thirties, recorded the album here last year, with Spoon producer Mike McCarthy (White Rabbits are often compared to Spoon, and Britt Daniel of the band even produced It's Frightening). They rented a house for three months, recording five days a week in the studio, jamming in their garage at weekends and hitting the same bar every night. It was a back-to-basics approach that they all appreciated.

Having formed in Missouri in 2004 when Patterson and Roberts met in college, they brought in various friends and former bandmates to make up the rest of the band. They moved to Brooklyn a year later and found themselves all living together in the same loft where they'd also practice. "It gave us the ability to always be on top of each other and know exactly what that's like all of the time," continues Clark. "But we found that we work better in small groups and so the majority of the time one of us will come with a verse or something and the band gets together and tries to flesh out an idea."

They certainly have the camaraderie of a band that was formed through friendship and one that has spent a considerable amount of time in close quarters. They are an affable bunch who share jokes and prompt each other when necessary.

And while they take their band very seriously, they don't accept that they are esoteric, as some critics have suggested, and Patterson is keen to tell me that while they might have been listening to a lot of Devo and T Rex while they recorded the album, they were also listening to a lot of pop radio, hip-hop and R'n'B, especially Beyoncé, who he is a big fan of. Another unlikely influence is Kanye West. "In my opinion, Kanye West has a lot more in common with Mick Jagger than any other rock'n'roller right now. He has this swagger and attitude about him. Outside of simply a musical influence, I think I found that listening to music that had such confidence was inspiring. Any music with some attitude. It's strange because confidence and attitude almost seem to be frowned upon now."

While the band has received many plaudits, curiously they remain somewhat under the radar. Fellow experimental Brooklyn bands such as MGMT or Yeasayer, for example, have managed to break though in a way that White Rabbits have yet to, despite consistently producing great albums and impressing with their live shows. Have they thought about this? "You have to be real about it and realise that everybody doesn't like everything," says Clark.."

"I mean, we're inherently ambitious, even our line-up is ambitious, so, sure, there's an element of wanting more," says Patterson. "I think about it. Especially when you're working really hard and you're poor and things don't come together as quickly as you want them to. We're still in that place where we play some tiny club and we can barely fit onstage. That being said, I think things are going great and I'm happy where we are."

One thing they don't tend to ruminate on is the future. "There is no big plan set out," shrugs Clark. "Bands with mission statements weird me out. We just want to play some great shows, make another album. That's kind of the drill. And, I guess, have a few beers."

White Rabbits play XOYO, London EC2, on 1 May. 'Milk Famous' is out now on Mute (whiterabbitsmusic.com)