In the shadow of the Shard: Why the next generation of skycrapers is struggling to get off the ground

Even as the mighty Shard on the banks of the Thames shoots towards its opening in May, plans for tall buildings from London to Liverpool are being knocked down as opposition grows to the tall, the ostentatious, the showy and "iconic". And the people knocking Britain's new towers back are not necessarily traditional conservatives and architectural purists, but young activists and many people usually associated with the avant-garde.

Last year, architect Amanda Levete's 25-storey tower in Shoreditch, east London was set to rise from the Huntingdon Estate, overlooking downtown's favourite edgy backwater, Redchurch Street, but was knocked back by a group of highly motivated residents including a clutch of high-profile artists, Misses Emin and Whiteread among them. More recently, a plan for a so-called eco-tower next to Dalston's Kingsland station – the hipster hub of east London – has just been repelled until further notice. With 130 flats, and fashionable accoutrements such as vertical gardens, it was designed by architects BFLS, which designed the now-familiar Strata tower in Elephant and Castle. In south London, two towers in the Vauxhall Cross development have been nixed for now, following ardent campaigning.

And the pro-low phenomenon is not just restricted to London. In Liverpool, the Liverpool Waters development, currently the UK's biggest planning application including a scraper of considerable height, has been set back following local opposition and has even raised the objection of Unesco (the city's Maritime Mercantile City is a World Heritage Site), which cited the possibility of a "serious loss of historical authenticity". There has been similar disquiet in Scotland, at the second phase of high-rise developments in the Glasgow Harbour site.

In one sense, these buildings are seen as out of time – signed off in the boom, built in the bust – and also out of place, particularly in classic four-storey Victorian streetscapes such as Dalston's. "It is inappropriate to the area, and full of 'greenwash'," says Bill Parry-Davies of Open Dalston, a group that campaigned against the tower, who says that nearby Waltham Forest has also had a similar application beside its station, indicating a trend. Parry-Davies is pleased, therefore, that Hackney Planning Committee recently rejected the exclusively private 18-storey tower block, where apartments rose from about £400,000 into six-figure sums.

Part of the wider problem is that whereas in the past tower blocks were for the poor, these iterations are mostly for the luxury market, aiming to attract investment money from overseas and therefore divorced from locality. "Tall buildings are socially divisive," says Michael Ball of the Waterloo Community Development Group, which has recently acted against two towers in the Vauxhall Cross area: Bondway, by Make architects, and the so-called Kylun. "The only people they please are developers and some of the architectural lobby." Recently, Sir Richard Rogers said London should build up, and that the Shard wouldn't be worse were it to be "10 storeys higher... It's the human scale of buildings that makes them work, not the height or width." Which is fine if it is responsible, but, as Ball adds, such developments are often there to soften up areas for further planning applications. If they fail, then something else will work and in any case, the site will appreciate from the planning work done. "It's happened at the site of the old Sainsbury's building in Blackfriars," Ball says. Hence the well-founded distrust.

The new generation of residential blocks is being delivered on the promise of creating greater density, to follow the mantra of "build up, not out", so local authorities are more likely to sign off tall buildings if they make the right environmental noises. "This equation is just not correct," Ball says. "The densest part of London is St John's Wood, because of mansion blocks. This is the architectural type that is going to create more housing for more people." In any case, he adds, most of the new blocks are for luxury flats, so they don't even have the benefit of the earlier generation of tower blocks, which at least attempted to create housing for the working classes.

But is this a new kind of neo-nimbyism, dressed up in social concern? The book Triumph of the City by Edward Glaeser, a favourite of British metropolitan authorities, suggests that objectors are a modern manifestation of the arguments developed by the seer of urbanism, urban-planning activist Jane Jacobs, driven to protect local areas against developers but ultimately raising local land values to their own benefit. "There can be an urban future where more people live in central cities, but to do that, the most desirable of those cities must reduce the regulatory barriers that limit the construction of taller buildings," he writes. It's a moot point, but this argument, along with the supposed ecological and density benefits of taller buildings, constitutes the more progressive of the counter-claims. "Maybe there's an argument that the recent generation of tower blocks are for the rich," says James Newman of the website Skyscraper News. "But I'd argue that it's important to have inclusive design, and in some ways, the newer buildings often do, with street-level activity designed in, as well as restaurants. The Shard even has a non-denominational meditation suite at the top. Tall buildings should be participatory as well as good-looking." Newman cites One Brighton, a high-density development in the seaside city with rooftop allotments, as a good example.

Developers and estate agents tend to concur, and make the argument more forcefully yet. "The arguments for such policies [of allowing tall residential buildings] are compelling," says Nigel Abbott of the estate agents Cluttons. "Taller buildings maximise available space in congested cities, and as we are suffering a severe housing shortage, it's little wonder that there are plans to build into the sky." And the way to make them work is to insist on good design and consideration to surroundings, Abbott says. "Properties then stand a better chance of being given the green light and will help bat-back those who oppose development," he adds.

Some in the architectural world agree, with the proviso that good design sugars the high-rise pill. "Most people are by nature nimbys, and anything tall seems to be the ideal target for that sort of criticism," says William Murray, director of Wordsearch, an architectural marketing agency. "I don't think that objections are necessarily a bad thing, if they can be channelled constructively into providing better buildings. We also mustn't assume that everything tall, new and shiny is necessarily good." Some are excellent, says Murray – he cites Feilden Clegg's Broadcasting Tower in Leeds – and some are bad. "For example, I think the London skyline would be a lot better off if there had been a few more objections to the Broadway Malyan St George's Wharf Tower and the Strata in Southwark."

Bob Robinson, chairman at planning company DPP, says that the recent objections are the latest in a process of disenchantment that has been gathering pace since the mid-2000s, and which has since been proved demographically wanting. "There was a boom then that led to the towers in Manchester and Leeds, some of which are in distress", Robinson says, adding that this housing model targeted one- and two-bedroom flats that failed to cater for wider family need. "Buyers are now trapped in these things," he says. "They get into their thirties and want to move out or bring children up somewhere at street level, and can't sell them."

More generally, Robinson reckons that tall buildings are having a hard time because they're simply not part of this era's mood music. "They're seen as ego statements, part of the mega-bonus culture. They are seen as wrong for the time." There's even a theory called the "skyscraper index", advanced by economist Andrew Lawrence, that tall buildings are harbingers of economic doom. Ominous and vulgar? No wonder people want to cut tall buildings down to size.