Raheem Sterling put City a goal ahead inside seven minutes after finishing off a wonderful one-touch move before Gabriel Jesus missed the chance to double their lead before the break from the penalty spot.
Mario Pasalic pulled Atlanta level almost immediately after the restart before substitute goalkeeper Claudio Bravo was sent off ten minutes from time.
Walker came on and took the gloves for the final moments of the match as City held on for a 1-1 draw in Milan.
A win would have secured them a place in the last 16 of the Champions League, while a draw puts them five points clear at the top of Group C and saw Atalanta earn their first point.
The match took place at the San Siro, home of both AC and Inter Milan, due to Atalanta's Gewiss Stadium in Bergamo not meeting Uefa requirements.
Pep Guardiola named a strong side before kick off with only record goalscorer Sergio Aguero rested with a view to Sunday's crucial Premier League clash with Liverpool.
Jesus started up front for the Cityzens with Sterling, Kevin De Bruyne, Bernardo Silva and Riyad Mahrez all included. Fernandinho was preferred to John Stones in the centre of defence.
Hans Hateboer came close inside five minutes for the home side. The Atalanta wing-back should have done better at the back post, but shinned his volley into the side netting.
Sterling made them pay for that miss less than two minutes later, giving City a foot in the door of the last 16.
Bernardo Silva played the ball in to Jesus from the left, who’s flicked back heel fell into the path of the 24-year-old who finished off a beautiful move into the corner.
The England forward almost made it 2-0 following a mistake by Atalanta, who gave the ball away in a dangerous area. Mahrez played a perfectly weighted ball in from the right, but Sterling just couldn't get there.
The away side were awarded a penalty after Sterling was played into the area and completely taken out by Rafael Toloi, but after a VAR check the decision was changed to a free-kick, with the foul occurring just outside the area.
From Sterling’s resultant set piece, his effort struck Josip Ilicic’s arm in the wall and the video assistant referee was back in action. This time calling on-pitch referee Aleksei Kulbakov to his monitor to check and he awarded a penalty.
After a peculiar couple of minutes, Jesus stepped up to take it but, trying to place his effort into the corner, side footed it about five yards wide.
City suffered a potential blow ahead of their clash with Liverpool on Sunday as Ederson was replaced by Claudio Bravo at half-time.
They suffered another not long after as Atalanta pulled themselves level four minutes after the restart through Chelsea loanee Mario Pasalic.
Alejandro Gomez whipped in a beauty of a cross and Pasalic met it with a bullet header leaving Bravo no chance.
Things got from bad to worse on the goalkeeping front for City. With 10 minutes to go Bravo, who only came on at half-time, was sent off for taking down Ilicic.
With no other goalkeeper available after a lengthy delay, Kyle Walker came on from Mahrez and took the gloves for the final moments of the match.
Walker was immediately in action facing the free-kick. It came in under the wall, he spilt it but recovered well to make the save.
City held on for the draw with Walker in goal, and remain top of their group in the Champions League.
Featured from our writers