Budapest has a new and distinctively odd landmark. I am peering down from a window of the presidential suite on the ninth floor of the Kempinski Hotel Corvinus in Pest. Across the road, in Erzsebet Ter (Elizabeth Square), there is a tree that appears to be bandaged in paper.

There is a rational explanation for this. The hotel management has granted me privileged access to the inner sanctum of a temple. This suite was Michael Jackson's residence whenever he was in town. Though it has been redecorated since his final visit, the Versace-influenced furnishings and the whole 189sqm extravaganza is every bit as blingy as one would hope. However, for the Jackson faithful, the inner sanctum remains unattainable; the nearest they can get is the tree in the park. Since his death, fans have festooned it with photos, magazine cuttings, fresh flowers and billets-doux addressed to their idol. It is an improvised but semi-permanent shrine.

It has yet to become a major draw but, as Budapest's sites go, it has the benefit of novelty. Most of the others have become so familiar to me over the years that I tend to underestimate their impact on first-time visitors. Being part Hungarian, I feel at home here, but maybe I take my birthright slightly for granted. A part of me envies the first timer. Like them, I want the thrill of discovery. I want to feel Budapest emerge afresh in my imagination as one of the great cities of modern Europe. I want to be a born-again tourist.

I resolve to do as the best tourists do in a new city: walk. Most of the big hotels are concentrated in a small area of central Pest known as Belvaros (the Inner City; municipally, the Fifth District) and most of the principle landmarks are within strolling distance. Exploring at street level reveals many small surprises. It hadn't occurred to me before, but as I walk past the ornate secessionist gates of the Gresham Palace (now the Four Seasons Hotel), I realise there is a British axle at the hub of Budapest.

The palace, one of the most graceful Art Nouveau buildings in the city, was commissioned by the London-based Gresham Life Assurance Company. It occupies a prime location in the square on the Pest side of the famous Lanchid (Chain Bridge). This, too, has a British provenance. The first permanent stone bridge in the city was designed by an Englishman, William Tierney Clark, and is a grander version of one he built earlier in Marlow, Buckinghamshire. Moreover, the square on the Buda side of the bridge is named after the Scottish engineer Adam Clark (no relation), who oversaw its construction.

Walking across, you notice the scale of Budapest. This is no twee medieval museum town in east-central Europe. It has ambition. Perhaps that is hardly surprising from a former capital of the Austro-Hungarian Empire. It is, in fact, bigger than the other capital, Vienna, and marginally more populous.

This morning, the river is more the Grey Danube than the summery blue of Strauss's corny old tune, but, even in winter, under a sky leached of colour, you cannot fail to appreciate the beauty of the topography. The scenic Buda hills on my left descend in a series of humps and escarpments to the drifting north-south line of the river and, on the opposite east bank, lies the commercial and administrative heart of the country in Pest – with a crowded skyline all the way to the flat horizon.

To appreciate these views, the next best thing to a helicopter is the funicular railway that starts in Adam Clark Square. The Budavari Siklo was originally intended as a quick and cheap commute for workers employed in the Var (Castle district) at the top of the hill. It is now entirely a plaything for us tourists. The quaint wood-panelled carriages look like wonky but upmarket garden sheds, and seem in extremely good nick for such an antique mode of transportation. Their near pristine condition is explained by the fact that the original machinery of the 1870 Siklo was destroyed during the Second World War. The current railway is a replica that opened in 1986. The ride lasts just 90 seconds and, at 1,000 forints (nearly £3) per adult, it cannot be classed as the cheapest thrill in town.

The Castle district offers some of the city's most pompous architecture – a procession of palaces, churches and monuments – and is a must for any Budapest neophyte. The changing of the guard I witness is a rather homespun affair, with strutting soldiers struggling to hang on to some sense of decorum amid the camera-clicking hordes.

The area took a terrible beating during the siege of Budapest, a vicious street fight that lasted six weeks and which has been compared in its intensity with Stalingrad. The Germans (aided by Hungarian fascist allies) made their last stand against the Red Army here. Much of the architecture has a recently restored feel: the edges seem sharper, the colours brighter and the façades smoother than one might expect. The makeover is so convincing that, when I round the corner into Disz Ter, the sight of the shattered former Ministry of Defence, which has been left as it was at the end of the war, comes as a shock.

Just a few hundred metres away, the 19th-century neo-Gothic splurge of the Matthias Church and the peculiar Fishermen's Bastion looks even more Disneyish after such a visceral reminder of recent cataclysm. The turrets of the "bastion" resemble a series of elves' hats strung out on a monumental washing line. Snow White and all her dwarves would not look anachronistic proceeding along the fairytale terraces, but such reveries are interrupted by the megaphone voices of German and Japanese tour guides marshalling their flocks. Bemusing though it is, the Fishermen's Bastion does nevertheless offer definitive panoramic views of Budapest.

Round the corner from my hotel in Pest, I find another survivor of the many horrors inflicted on Budapest in the 20th century. The Gerbeaud Confectionary, in Vorosmarty Ter, opened its doors in 1870 and reached its apogee in the belle époque as a temple of sweet indulgence. I remember being brought here as a child in the dour decades of communist rule, and yet, through all those years, despite suffering the indignity of a name change, the café everyone still called Gerbeaud retained a whiff of its original decadence.

Trading under its old name again, the café has been restored to its full former glory by its new German owners. The quality of fare has gone up – and so have the prices. My friend Celia, who is working in the city for a few months, is a fan. "I love the apple strudel. It is soooo good. The best!" she enthuses. But she concedes the pricing is eye-watering for many Hungarians. "It's not a local place, which is really sad."

The Lotz Café, in the former Paris Department Store, is more reasonably priced but reason has little to do with anything else. The Art Nouveau frontage of the building, on Andrassy ut, is eye-pleasing enough but it gives little hint of the neo-Baroque glory hidden within. Nowadays, the building is principally a bookshop. To reach the café, take the escalator away from the volumes on the ground floor – without warning, you are propelled into a ballroom that Marie Antoinette would not carp at. There is more than a hint of Versailles about this grandiose interior. Huge mirrors reflect and amplify the ostentatious gilt-edged decoration featuring frescoes by the 19th-century artist Karoly Lotz. It is wildly over the top for what is, after all, only a coffee shop.

The room was intended for an adjacent casino in 1885, before being knocked through to become a part of the department store. After the Second World War it was neglected, serving as storage space for such workaday goods as were available in the communist-managed economies of the Eastern Bloc. Today, reinvented as a café, it has a wicked glint to it. Remarkably, for all its outrageous posture, the down-to-earth prices in the Lotz make it a favourite for locals and tourists alike. It is a worthy addition to café society in a city famous for its coffee-and-cake culture.

I have one last obligation as a born-again tourist. Tramline No 2 has been named the seventh most scenic in the world – and the top in Europe – by National Geographic. It is an accolade of which Budapest's natives are proud, but one that many feel only confirms what they already know. The ride begins unpromisingly as it departs its southern terminus and rolls past the cultural complex next to Rakoczi Bridge. The Palace of the Arts is a fine building but it suffers from its proximity to the newish National Theatre, an alarming heap of terrible ideas, next door.

The tramline heads north on the Pest bank of the Danube and normal service is resumed. Gellert Hill, with its giant post-war statue of liberty, dominates first, and many of the other wonders of Budapest follow in swift succession; the bridges, the hills, the baths, the castle and the broad sweep of one of Europe's great waterways. Then the outline of the Palace of Westminster seems to appear out of the low winter mist. This is not a hallucination. The Gothic Revival style of the Hungarian parliament was indeed inspired by Westminster and its river-front position adds to the illusion that we are some 1,500 kilometres to the west. But there is a further twist.

This parliament is almost the same size as its antecedent in London – which is pretty cheeky for a small landlocked nation. Some would say that's typical. The Hungarians were not imitating to flatter, they simply wanted a building commensurate with their self-image.

Travel essentials

Getting there

Sankha Guha travelled as a guest of British Airways (0844 493 0758; ba.com/budapest), which offers three nights at the Kempinski Hotel Corvinus in Budapest from £299 per person, based on two people sharing. The price includes return BA flights from Heathrow and breakfast.

Alternatively, Ryanair (0871 246 0000; ryanair.com) flies from Birmingham, Bristol, Manchester and Stansted; Jet2 (0871 226 1737; jet2.com) from Edinburgh and Manchester; easyJet (0843 104 5000; easyJet.com) from Luton and Gatwick; and Wizz (0906 959 0002; wizzair.com) from Luton.

More information

budapestinfo.hu