Goals from Troy Brown and Dani Lopez were enough to make it two league wins on the trot for Aldershot against relegation rivals after they beat Barnet last time out.
Defeat for Accrington was their fourth league reverse in their last five games and Richardson admitted his side lacked the Shots' cutting edge.
"We had opportunities to put corners and crosses in, but we didn't take it and they showed us how to do it at the other end," Richardson said.
"They were better in the areas that matter, putting the ball in the box and attacking it, but there wasn't that much in it.
"I'm very disappointed because we started very well.
"We got into some great positions but didn't quite have the composure to finish them off and then on their first bit of threatening play they score from it.
"You find yourself 1-0 down and you think, how? We started very well, for an away side, and there was nothing between both sides.
"I am frustrated, but we knew before that we have a tough period coming up, it's the same for everybody.
"We all knew it was a big game, a six-pointer, but we've got to remain positive and we will."
- Sports & Recreation