Coming into the match on the back of six straight Sky Bet League One defeats, it was left to James Dunne to breathe life into the visitors' season in the second minute of time added on.
Victory came as a major relief for Westley, who won his first game back in charge since returning to the club for a second time in March.
But the former Preston boss revealed there was no broken crockery during his half-time team talk.
"We won't get carried away with the win, but I'm pleased with the way we went about our business," he said.
"We didn't give anything away and it was a robust and solid away performance.
"We know what it takes to get results in this league. We've got a number of young players and it will take time to gel.
"But in terms of the scare-o-meter, I was pleased with the whole defence. They all played their part in getting a clean sheet without too much fuss.
"I was able to drink my half-time cup of tea without spilling any. Then I punched the air as hard as anyone when the goal went in. It is good for the whole camp to get those three points.
"I'm sure that going to Everton next in the Carling Cup will be very different. It's a tough place for Premier League clubs to go, let alone a club of our stature."
Stevenage's Robin Shroot was denied by a flying save from County keeper Bartosz Bialkowski after 36 minutes and was foiled by his legs after a flowing move seven minutes after the break.
County's Callum McGregor and Shroot both shot over the crossbar in the closing stages of the first half before Stevenage grew in confidence after the break.
But it took until injury time when Shroot's endeavour down the left gave Dunne the simplest of finishes.
- Sports & Recreation
- James Dunne