Serbia, who won their group and were playing the lowest seed from the opposite pool narrowly avoided a huge upset. They trailed Australia right until the final four minutes of the match, before winning 11-8.
Montenegro beat Spain 11-9 in an earlier quarter final clash to book into the semis, giving the country a shot at winning its first Olympic medal as an independent nation.
"I was afraid," admitted Serbia's Filip Filipovic said of how he felt when they were behind, but he said he took confidence from the team's ability to turn the match around.
"I think that this team showed spirit. When we play badly like in the first two quarters, we can rise up again, and we can play the most beautiful water polo."
The team roared back to life in the second half of the match with a torrent of goals from their three top scorers, Andrija Prlainovic, Filipovic and captain Vanja Udovicic, in a display which saw Serbia's famed defence recover to put a stop to Australia's run.
Serbia, who won bronze in Beijing, have spent the past four years on a roll, winning every major title on offer and are favourites to win the tournament after an unbeaten run so far.
Montenegro cruised through the middle periods of the game before letting a four-goal lead slip in a tense fourth quarter as Spain capitalised on their extra-player situations.
Montenegro narrowly lost out on the bronze to take fourth place in Beijing, when it competed in its first Olympics as an independent country since it separated from Serbia in 2006.
"I don't want to be one more time fourth, I want to take a medal. It's very important for us to take a medal," Montenegrin captain Nikola Janovic said after the win.
The team was cheered on in the stands by Prime Minister Igor Luksic earlier on in the tournament, who spent his holiday in London watching the country's teams compete in water polo and handball, such is his desire for a medal for Montenegro.
"We must be a little crazy. It's the moment. It's one moment (of) inspiration," said Janovic when asked how his team will win their next match and guarantee a shot at the gold medal.
Montenegro will play either Croatia or the U.S., who meet in a quarter final match later on Wednesday, in the semi-finals scheduled for Friday, while Serbia will meet either Hungary or Italy.
The Serbians have already overcome defending champions Hungary, looking to win a fourth consecutive Olympic gold, in the group stages.
"Doesn't matter. Semi-final, tough game, everybody comes here to win," Serbian coach Dejan Udovicic said when asked which team he would rather meet in the semis.
The former Yugoslav nations of Montenegro, Serbia and Croatia all play a similar style of water polo, which has to date help them dominate at this year's Games.
For Spain, who last got a medal in the water polo in 1996 when they took home gold, the loss was a painful repeat of 2008, when they were also defeated in the quarter-finals.