The Tuscan outfit started brightly and had almost total control of the ball with Adem Ljajic getting the best opportunity of the opening exchanges.
Ljajic pounced on a David Pizzaro pass in the area to force a smart save from his fellow-Serb Zeljko Brkic.
However, it was the impressive Reginaldo Maicosuel who opened the scoring for Udinese against the run of play after 28 minutes, by carving open the Fiorentina defence with a smart diagonal run.
He met Luis Muriel's excellent through ball and slotted home first time with his left foot.
The pattern continued in the second half, with Fiorentina dominating possession and Udinese defending stoutly.
Fiorentina made the breakthrough after 67 minutes when Stefan Jovetic's deflected shot looped over the already committed Brkic.
Brkic nearly palmed a cross into his own net and then, with 17 minutes left, new signing Mounir El Hamdaoui, who had come on for Ljajic, shot wide at the near post from a tight angle.
When Manuel Pasqual's free-kick crashed against the post with three minutes left, it looked as though Fiorentina would have to settle for a draw, but Jovetic popped up in the last minute to curl home the winner and grab all three points.
"It was a great feeling, scoring right at the end," said Fiorentina manager Vincenzo Montella.
"The team stayed together even after going behind and was clear-headed enough to bring the result home.
"Even if we hadn't succeeded in turning the result around, then they would have still said that we played well."
- Adem Ljajic