Gary Roberts and Eoin Doyle marked their home debuts with a goal apiece but goalkeeper Tommy Lee also played his part when Cheltenham fought back in the second half.
The stopper made a crucial block to deny Byron Harrison an equaliser prior to Doyle's late goal.
Yates said: "I thought for 22 and a half minutes we were doing really well. We had a game plan and although they are a good side, we were stifling them and we were just getting into the game and creating chances when we get caught on the break.
"I thought we were much better in the second half, we played with a bit more urgency and we were the better side and we were a bit unlucky to have a second goal against us to be honest.
"We could easily have had a point, Byron had the best chance in the first half when he dinked the keeper and the lad cleared it, perhaps he could have done a little bit better.
"We had a goal disallowed, we had a penalty shout that the boys are adamant was and Byron danced his way through and was a bit unlucky but that's how it is sometimes away from home and we have just got to keep working hard."
- Byron Harrison
- Eoin Doyle