The announcement in a packed Quebec City ballroom brought smiles but few cheers from the surviving cities as all three know the challenges that lie ahead after having previously cleared the first hurdle with recent bids.
"It is the third time we have been invited to participate in the final run and we are very excited to bring this to Spain," Spanish IOC member Juan Antonio Samaranch Jr., son of the former IOC president, told Reuters.
"This is a kind of Olympic competition with only a gold medal. We have bronze, silver and now we go for the gold.
"My father would not waste one second wasting how happy we are, he would have us all meeting in a room right now thinking what's next."
Next for Madrid, Istanbul and Tokyo is 16 months of furious campaigning as they try to convince the IOC they are up to the task of staging the world's biggest sporting spectacle.
The high stakes run-off reaches the finish line in Buenos Aires in September 2013, when the IOC will vote on the 2020 host city.
Doha and Baku head back to the drawing board.
Flush with cash from natural gas reserves, Azerbaijan had invested more than a billion dollars in sports venues in the past 10 years and is likely to try its luck again in 2024.
Oil rich Qatar possessed the financial muscle to carry off the 2020 Olympics but the tiny Gulf nation could not convince the IOC to take a chance on holding the Summer Games in October.
It was not just the threat of searing temperatures that frightened away the IOC but also the prospect of low television ratings if the Olympics was to go up against other major sporting events.
The IOC made it clear in the evaluation report that holding the Games in October was a non-starter.
"In July/August, people have more leisure/vacation time. There is therefore a risk that an October Games would become a 'weekend Olympics Games' and with a reduced demographic reach, broadcasters would have difficulties in attracting the same audience levels in terms of working people and youth."
If Doha's bid team were consoled by Qatar's successful bid to stage the 2022 FIFA World Cup, they were not showing it.
"I think the two countries that were eliminated are emerging markets and I think it is a missed opportunity for the IOC," said Al Mayassa Al-Thani of the Doha bid.
"The traditional dates don't suit Doha and that is nothing we can change and we provided the dates we thought were most suitable.
"We believe we did everything in our capacity and in the end it was a decision of the executive board we have to respect despite our disappointment."
With the global economy in upheaval, selecting a 2020 host represents a tricky decision for the IOC as it looks eight years into the future and ponders a fuzzy financial picture.
Of the three remaining bids, Madrid faces the toughest financial questions because of Spain's troubled economy.
The IOC report praised Madrid for its "clear vision" and high levels of government and public support but warned "careful attention would need to be paid to Spain's economic outlook".
Tokyo, which hosted the Games in 1964 in the days before the current drawn out bidding process, will have to convince voters to bring the back-to-back Olympics to Asia with PyeongChang already set to stage 2018 Winter Games.
There will also be questions over whether the country is capable of taking on another massive challenge as it continues to rebuild from the tsunami and earthquake that devastated much of Japan last year.
"The dream among the people who are facing the biggest difficulties in the whole world, we really believe the power of sport encourages people," said Tokyo bid chief Masato Mizuno.
"So if we make a really super good Olympic Games this inspires the whole world and the people in Japan."
All three cities have bid before but Istanbul, bidding for a fifth time, could finally be rewarded for its persistance.
Bridging Europe and Asia, Istanbul is seen by many as an attractive choice but Turkey's pursuit of football's 2020 European championships continues to provide a distraction that the IOC said presented "significant risks".
"We are here for the Olympics today, this is our priority," said Istanbul bid leader Hasan Arat, dismissing questions about Turkey's pursuit of the European championships.
"There will be one Games, one city, two continents - Istanbul offers a lot of things."