Two days after Magpies chief executive Jim Rodwell sacked Chris Kiwomya, County picked up their third league win of the season to move off the bottom of the table.
Danny Haynes nodded them ahead after 19 minutes, only for Adam Rooney to level things up on the stroke of half-time with a header of his own.
Oldham's Jonson Clarke-Harris smashed home a free-kick soon after, but County clung on for a valuable victory.
"This is just one win, we need to string together a few wins now and get out of this situation," said Hodge.
"Mr Rodwell asked me to try and get a result, the players have got this result.
"We have to stick together and they did that and ground out this victory.
"It's great for the young players to get a victory. Young Callum (McGregor) is a technically excellent player and he will go on to bigger and better things.
"I have had to change my team sheet four times, I have lost five players before the game with either injury or illness.
"You never know whether a young player is good enough until you throw them in, and Jordan Holt did very well on his debut, he always shows a great calmness in his game and he did that.
"We gave ourselves a tough 20 minutes conceding the second goal, but we dug in and got ourselves a terrific result."
Oldham boss Lee Johnson found few positives from his side's performance, other than the glimpses shown by Clarke-Harris.
"Overall no complaints, other than with my players," he said.
"You realise how you miss players at times. When you have kids out on the pitch and they play like babies you get nothing out of the game.
"We were so lucky to be one-all at half-time. We deserved nothing from the game on the basis of that first-half performance.
"I was thinking of making three substitutions after half an hour, we were that poor.
"We conceded two poor goals in the second half, standing way too far off their players and giving them way too much space.
"I am a bit angry and can't really talk to anyone in the dressing room. I'd end up having a fight with one of them.
"Clarke-Harris did well at times, he does a bit too much of this dramatic falling about at times, but I'll have a word with him about that."
- Sports & Recreation
- Chris Kiwomya