The Gulls conceded the only goal of the game after an eight-minute stoppage while former Swindon striker Elliot Benyon, who had come on as a substitute, received treatment after being knocked out.
Torquay boss Alan Knill said: "We got done by the goal soon after Benyon's injury. It was down to a lack of concentration. Elliot is all right but not as chirpy as he is usually.
"We are still a work in progress and it took us 20 minutes to get our game plan working."
Andy Williams struck the winner seven minutes from time at the end of a closely-contested match which looked destined for an additional 30 minutes.
Swindon, though, should have gone ahead in the 56th minute when Alex Pritchard was brought down by Lee Mansell in the penalty area, only for Gulls goalkeeper Martin Rice to push his resulting spot-kick away to safety.
And Knill reserved plenty of praise for Rice after being handed the chance to hold down the number one spot.
"Rice is developing all the time and deserved this first chance to become our regular," Knill added.
"I was not sure it was a penalty at first but when the goalkeeper saved it, it looked as if he had set it up for us to go on and win.
"I was delighted with the whole team, the work-ethic, but just gutted about the result."
- Sports & Recreation
- Alan Knill
- Elliot Benyon