Dunne struck in the second minute of injury time to hand Boro their first points of the season and only their second win since Graham Westley returned to the club for a third time in March.
The striker saw his right-footed shot denied by a flying save after 36 minutes and was foiled by Bialkowski's legs after a flowing move seven minutes after the break.
Stevenage goalkeeper Chris Day was untroubled throughout a scrappy 90 minutes as Notts failed to capitalise on a string of first-half crosses from David Bell.
County's Callum McGregor and Shroot both fired over the crossbar in the closing stages of the first half before Stevenage grew in confidence after the break.
Michael Doughty, son of the late Forest owner Nigel Doughty, had a composed game in Stevenage's midfield and the visitors pinched the points when Dunne slotted home from close range after Shroot's work down the left.
- Sports & Recreation
- Robin Shroot
- Bartosz Bialkowski
- Graham Westley