The last time the Commonwealth Games was struggling to find a replacement host, after Durban was stripped of the event in 2017, it had help from an unlikely source: Buckingham Palace. An insider picks up the story. “The Palace went to the government and said: ‘It’s the Queen’s jubilee in 2022, you need to do something,’” he tells the Guardian. “UK Sport also then got a call. There was genuine pressure and that made Birmingham happen.”

On Monday the Commonwealth Games lurched towards yet another existential crisis after the Gold Coast withdrew its bid for 2026 – only four months after Victoria also pulled out. This time, though, there will be no call from the palace and the UK government riding to the rescue with £594m in its pocket. Just 16 months after the success of Birmingham 2022, organisers are again facing pointed questions about the event’s future.

Related: No UK rescue for 2026 Commonwealth Games after Gold Coast withdrawal

Some of the challenges it faces are longstanding and vertiginous. Even the world’s best PR agency, you suspect, would struggle to rebrand an event which began in 1930 as the Empire Games for Britain’s colonies. And hosting a sporting event with more than 5,000 athletes is getting increasingly expensive. Birmingham 2022 cost an estimated £778m. Victoria 2026 was forecast to be four times that when organisers pulled the plug.

It doesn’t help when TV revenues are a fraction of the Olympics and Fifa World Cups, and when the threat of terrorism lingers. Security costs for the 2000 Olympics in Sydney totalled $250m (£198m). By Athens, four years later and after 9/11, they topped $1.6bn (£1.25bn).

To make matters tougher for organisers, economists have frequently found the supposed benefits of hosting mega‑sporting events are overstated. Before hosting the 2002 Winter Olympics, for instance, Utah’s state government predicted it would generate 35,000 job‑years. However, after crunching employment data from 1990 and 2009 in Utah, the economists Robert Baade and Victor A Matheson found “no identifiable increase in employment either before or after the Olympics”.

Their conclusion is worth noting. “Considering that the federal government spent $342m directly on the 2002 Olympics and at least another $1.1bn on infrastructure improvements leading up to the Games, this amounts to about $300,000 in federal government spending per job created,” they state.

“Indeed, these results lend credence to a common rule‑of‑thumb often used by economists who study mega‑events: if one wishes to know the true economic impact of an event, take whatever numbers the promoters are touting and move the decimal point one place to the left.”

Any future Commonwealth Games will surely have to be smaller and cheaper to survive. Assuming, of course, there is a future. Insiders insist preliminary talks have been held with four regions about hosting in 2026 and they will give an update in the new year. But crucially India, the biggest fish in the Commonwealth outside of the UK and Australia, does not appear inclined to take the bait.

There are two major reasons for this. First, there is still a lot of lingering resentment between India and the Commonwealth Games Federation after the Games in Delhi in 2010 were mired in allegations of corruption and financial mismanagement. A parliamentary report in India on those Games also found “complete management failure” within the organisation and said the Indian government “nearly defaulted” on staging the event.

More recently, India threatened to pull out of the 2022 Games over the absence of a shooting event. To most observers, Narenda Modi’s government is now more focused on a far bigger target – hosting the Olympic Games in 2036.

In fairness to the CGF it deserves credit for frequently trying to innovate the Games, including by introducing eSports in Birmingham. It can also point to the way the event attracts a wider variety of athletes from superstars to enthusiastic amateurs.

Yet it will be a challenge to find a host for 2026 in such a short time. And given most international sports federations have already booked in events for that year, attracting big names might be harder than usual. In Birmingham many stars – include the Olympic and world champions Andre de Grasse, Shelly-Ann Fraser-Pryce and Shericka Jackson – stayed away.

No wonder some close to the Commonwealth Games fear the worst. “I always thought the Games would continue until the centenary in 2030 and then stop,” says the insider who fondly remembers the day Buckingham Palace came calling. “But my genuine view is we have seen the last Games under this format. At best it will look dramatically smaller in size, scale and interest.”

Do you have an opinion on the issues raised in this article? If you would like to submit a response of up to 300 words by email to be considered for publication in our letters section, please click here.