FA Cup - Hart gaffe hands Leicester draw
Hart spilled a routine cross by Paul Gallagher on 64 minutes, allowing Andy King to convert the rebound for the game's final goal.
Debutant Sol Bamba had given Leicester the lead after 45 seconds with his first touch for the club, before goals from James Milner and Carlos Tevez gave the visitors a half-time lead.
The match pitted Leicester boss Sven-Goran Eriksson against Roberto Mancini, who played for him at Sampdoria and Lazio.
And the Swede took the upper hand in the first minute when Ivory Coast international Bamba - signed earlier this week from Hibernian - turned in a corner to send the 32,000-strong crowd into raptures.
Mancini eschewed his usual blue and white scarf for a red and black edition, echoing the colours worn by Manchester City when they beat Leicester in the 1969 FA Cup final.
And Mancini's side were on level terms on 23 minutes after a superb piece of play by Milner. The England midfielder jinked past Bamba and fired a low left-foot shot into the left corner of the net.
Tevez gave the visitors the lead just before half-time when City played a short corner into the box, and the captain flicked the ball deftly past Chris Weale.
City had problems of their own defending set pieces, however, and looked vulnerable at every corner. Bamba won a free header that forced Hart into a good save - then had another header on target from the rebound.
Elsewhere, Yuki Abe had a long shot turned behind, and Hart saved an attempted chip by King when he was clean through on goal.
Jack Hobbs powered a free header over from another corner, while Bamba had yet another effort headed off the line as the visitors' marking crumbled.
Leicester levelled on 64 minutes when the excellent Gallagher crossed right-footed from the left and the ball bounced up into the hands of Hart - but the goalkeeper fumbled the ball and King controlled the rebound before slotting home.
Both sides could have won it. Manchester City squandered chances through Adam Johnson and Nigel De Jong, while their inability to mark at set pieces left the door ajar for Leicester.
But an entertaining encounter ended two apiece, earning Leicester a lucrative replay at Eastlands.