Kieran Agard earned and converted the decisive spot-kick as the Millers stretched their unbeaten run to 13 matches at London Road in their manager's 100th match at the helm.
One-time Posh loanee Agard went down under a challenge from Ben Nugent in first-half stoppage time.
Already-booked Nugent escaped a second caution, but Posh were still punished as Agard dusted himself down to roll his 18th goal of the campaign past goalkeeper Bobby Olejnik to help Rotherham climb to fourth in the table.
Evans said: "This goes down as a big victory against a very good team, which arrived thanks to a terrific away performance.
"Kieran is adamant there was some contact which makes it a penalty kick and taking that into account you have to be a bit disappointed not to see a second yellow card.
"Peterborough's goalkeeper kept them in the game with a big save in the second half and our goalkeeper has made a couple of his own. When that happens and we win 1-0, he gets a little kiss from me when he walks into the dressing room!
"We have to remember nothing has been achieved yet and it is a case of continuing to work hard and making sure we get the play-off place we all want.
"I have no doubt that if we finish above Peterborough we will be in there."
- Sports & Recreation
- Kieran Agard
- Ben Nugent