James Baxendale's 25th-minute goal proved the difference as he fired home from 10 yardsafter Oldham's defence had failed to clear an Adam Chambers cross.
But Smith said: "We weren't at our best and certainly made it hard for ourselves.
"We weren't so good on the ball and looked a big lethargic.
"We didn't look like conceding and we were strong defensively. Oldham put us under a lot of pressure in the second half, but didn't really trouble us."
Smith, however, is delighted by Walsall's start to the season having won their opening league games against Tranmere and Oldham as well as recording a victory at Port Vale in the Capital One Cup.
"We have had three tough games in a week, but it's three points and a clean sheet away," he continued.
- Sports & Recreation