Harry and Meghan, or hostile environment: which is the real spirit of Britain?

The rest of the world is baffled by Britain – and understandably so. In the opening ceremony of the 2012 London Olympics, directed by Danny Boyle, we dramatised a confidence, openness and multiplicity that only a nation at ease with itself could muster. The global “soft power” of our language, cultural and scientific institutions, and international development programmes – still ringfenced by the Treasury – remains formidable. And on Saturday, hundreds of millions around the world watched the royal wedding: as glorious a fusion of pageantry and pluralism, of monarchy and modernity, as you could hope for.

Yet this spirit of generous self-assurance has its evil twin. We are also the country of Brexit; of Nigel Farage; of Jacob Rees-Mogg posing outside No 10 to deliver a petition demanding cuts to the aid budget; and, most recently and shamefully, of the Windrush scandal.

Which is the authentic Britain? Open or closed? Warm or frostily parochial? Both, I suppose. Most countries are, to use a word much-loved by Nick Clegg, “bicephalous”: two-headed and contradictory. The question we should ask in 2018 is: which of the two heads dominates?

On my travels in the past year, I have encountered understandable confusion about our national trajectory. To crunch a complex series of perceptions into a seriously oversimplified narrative, the view from beyond our shores runs as follows: Britain was in terrible shape in 1979. Margaret Thatcher’s shock therapy, painful for a great many, laid the foundations of renewed national prosperity. Labour’s election in 1997 marked a determination to match economic growth with social justice and liberalism.

According to this happy version of events, the 2012 Olympics represented the culmination of more than three decades of regenerative work: a global festival in which Britain declared itself to be both open for business and open in spirit. Into which narrative, the rusty spanner of Brexit was thrown with clanking ferocity four years later. Suddenly, Britain was declaring its furious hostility to “mass immigration”, to its putative imprisonment as a “vassal state”, to the supposed mutilation of its island heritage. Optimism had yielded to fear. We were, to quote Nigel Farage’s vile poster, at “breaking point”. You can forgive the rest of the world its perplexity. When, exactly, did Britain decide angry nativism was the way forward?

This global confusion about who we are and what we want has profoundly practical consequences. Ask any senior diplomat or chief executive and they will tell you the same thing: namely, that technical detail forms the structure of any deal, alliance or treaty, but that the much more nebulous question of culture, shared ideals and emotional identity determines the foundation. Trade agreements and investment decisions are not decided by algorithm alone. Each side must ask: what sort of partner am I aligning myself with? Will my employees be heading towards an outstretched hand or a “hostile environment”?

If I had to single out the principal cause of our national identity crisis, I would cite the catastrophic absence of leadership on immigration policy. The issue has maddened us, unnecessarily and tragically. It has drained much of our national confidence, made us mean, bred an uncharitable insularity.

In their words, deeds and tactical silences, politicians of both main parties have allowed a series of myths to enter and settle into mainstream discourse: that migrants depress wages and take jobs from indigenous citizens; that migrants are a drain on the taxpayer; that only the “metropolitan elite” benefits from migrant labour; that migrants are more likely to commit crimes. All this is rubbish, and deserves to be robustly and repeatedly dismissed as such – not left to fester in the national consciousness. I also believe immigration is culturally enriching. But then I have skin in the game (the surname is a clue). So let us concentrate on economic necessity.

A familiar refrain of pro-Brexit MPs is that their constituents were somehow “not consulted” on recent immigration. To which the only answer should be: oh yes they were. They were consulted when they voted repeatedly for an NHS that has the capacity to treat them – an objective that would be unthinkable without migrant labour.

They made a decision in favour of immigration when they opted for low-price consumer goods, for rapid delivery services, for facilities and shops that are open around the clock. They chose. We all did. It is outrageous hypocrisy to pretend otherwise.

In cabinet, I am told, even the arch-Brexiteers declare that national prosperity should not be sacrificed to the achievement of arbitrarily low net migration targets. Well, thanks a lot. Now they spot the flaw in the plan.

To be fair to the Brexit secretary, David Davis, he has repeatedly conceded that immigration policy post-Brexit will require many exemptions to meet the needs of the labour market. And as for all those shiny trade deals we are promised after 29 March 2019: the immigration minister, Caroline Nokes, told a Commons committee in February that visas could “absolutely” be exchanged for commercial access in future negotiations.

Brexit encapsulates this cognitive dissonance and its damaging consequences: it is about much more than our institutional relationship with the EU. I long for political leaders with the courage and candour to declare that the great challenges facing this country – automation, fiscal fairness, entrenched poverty, human longevity – have nothing to do with immigration, race or the much-needed influx of newcomers. That it has taken a royal wedding to reinstate this straightforward truth should shame our drab political class.

• Matthew d’Ancona is a Guardian columnist