Siddle, who finished the match with figures of 9-104, and left-armer Starc, who took 5-63 in the second innings, caught fire after tea to bowl the tourists out for 255.
Having been stymied in similar circumstances by a brilliant rearguard action from South Africa in Adelaide last month, Australia were desperate not to repeat the experience in Hobart.
However, it looked like it might be a case of deja vu at tea after some resilient Sri Lankan batting and a couple of rain breaks frustrated the hosts for two sessions.
Captain Michael Clarke, whose team will take a 1-0 lead to Melbourne for the second match in the three-test series, said he never doubted his bowlers.
"We knew we had to be patient," said Clarke, who was confident he would be fit for the second test despite a hamstring injury.
"No doubt I was confident that if we could bowl the balls in the right area for a long period of time that we would have success.
"It was going to be day's work to get so many good batsmen out but take nothing away from the bowlers, they had to execute their plans and their skills.
"If everyone's got the heart that (Siddle) has, we'll get back to being number one in the world no doubt."
Siddle had already taken the key wickets of captain Mahela Jayawardene (19) and ICC Cricketer of the Year Kumar Sangakkara (63) around lunch but the tourists reached tea on 186 with six wickets still in hand.
In a sign of Australia's growing desperation, wicketkeeper Matthew Wade shed his pads and gloves to bowl the final over before the break and claimed a maiden in his first spell in senior cricket.
Sri Lanka, set a victory target of 393, could still conceivably have won the match but a paltry return of 42 runs in the second session clearly indicated they were batting for a draw rather than a first win in Australia.
"Disappointed that we lost the game," said skipper Jayawardene.
"It was a tough test match in the sense that we were chasing the game probably after the third or fourth day but we kept fighting.
"Today we knew we had to bat through a couple of sessions sensibly, see what happens, and up until tea we were on target."
The Australian bowling unit, a man down after local quick Ben Hilfenhaus suffered a side strain, never gave up, even if Siddle looked like the only man who might break through for much of the day.
The 28-year-old struck again on the first ball of his second over after tea to dismiss all rounder Angelo Mathews for 19 and then removed the last of the Sri Lankan top order with a sizzling delivery which trapped Thilan Samaraweera lbw for 49.
With Sri Lanka reduced to 218-6, Siddle took a rest but left-arm quick Starc picked up the baton and had soon removed Prasanna Jayawardene for 21.
A fullish inswinger nearly took the wickekeeper's hand off and the ball flew off his glove to Mike Hussey at second slip to leave just three tailend wickets standing with around 90 minutes of play remaining.
Another peach of a delivery from Starc sent Nuwan Kulasekara back to the pavilion for nine and a perfectly delivered yorker soon afterwards removed Rangana Herath for eight.
With Siddle now bowling from the other end, it was a straight fight between the two for who would take the five wicket haul and win the match for Australia.
Starc took the honours when he had Shaminda Eranga caught behind for six, sparking huge celebrations among his team mates and the disappointingly small crowd of some 2,000.
"As a group, I think we've been outstanding since lunch on the second day and we got our rewards," said Siddle.
"I said the other day that I got lucky and got the results and someone else would get it in the second innings and Starcy did today."
Siddle was named Man of the Match but was also at the centre of a ball-tampering row after a picture of the bowler apparently picking at the seam in Sri Lanka's first innings circulated on social media.
The Sri Lankan team had not made an official complaint but spoke to match referee Chris Broad. The ICC later said no charges would be laid.
"Why would I jeopardise that when we can go out and win in the spirit of the game?" a surprised Siddle added.
- Sports & Recreation