The result provisionally lifted the Serbians, who have not played on the big stage since the 2010 World Cup, to the top of their group with 11 points from five games.
Second-placed Ireland have 10 points from four games and can leap back into pole position if they beat third-placed Wales, who have six points from four matches, in the evening kick off.
Georgia thoroughly dominated the first half and missed several chances after Nika Kacharava gave them a sixth-minute lead with his first international goal after a howler by Aleksandar Kolarov.
The Manchester City defender, heavily criticised after recent performances for the Premier League club, gifted Kacharava a sitter with an awful back pass and the striker beat goalkeeper Vladimir Stojkovic with a crisp finish.
Read the full article on eurosport.co.uk: Serbia put pressure on Ireland and Wales after win in Georgia