The hosts, looking to make the fourth round for the first time since 1998, were 2-1 up with just five minutes to go at Blundell Park after Ross Hannah and Craig Disley struck - sandwiched by Oliver Norwood's goal.
The visitors stormed out of the blocks and peppered the home goal in the opening stages.
First, Paterson forced goalkeeper James McKeown into action with an excellent header following a cross from the right.
Then Danny Ward's strike from 25 yards brought another smart save from the home keeper.
Paterson hooked another volley goalwards in the 13th minute to keep the young goalkeeper on his toes as the early domination continued for Mark Robins' men.
However, the Mariners served notice of their threat when winger Joe Colbeck surged along the right flank and fired a cross in which evaded everyone.
Grimsby went close again in the 19th minute when Thomas' strike was deflected wide for a corner.
And the Conference side were in front six minutes later when Colbeck again beat his marker and put in a low centre for Hannah to tap home from five yards.
Huddersfield responded and Ward's strike was superbly blocked by Paul Bignot before Adam Hammill tested McKeown again from 25 yards.
Grimsby defender Chris Doig then repeated the trick to keep out Ward's stinging effort as the hosts held on to their advantage at the break.
It took just six minutes for the Terriers to hit back, however, as Norwood lined up a free-kick from 25 yards and hit a stunning, curling effort past McKeown.
But on 62 minutes Grimsby striker Andy Cook saw his shot half-blocked and the ball bounced up for Disley to head home from 10 yards to restore their lead.
Hammill fired wide from distance as Huddersfield's frustrations mounted, before Thomas should have made sure of the result - twice.
First, the Grimsby left-back headed on target in the 75th minute, forcing Alex Smithies into a spectacular save.
Then, moments later, the same man headed wide at the near post from another set-piece.
But in the 85th minute the visitors were level when Duane Holmes' strike was parried away by McKeown and Paterson was able to side-foot home from 10 yards and level the scores again.
However, in the final minute of normal time, a low cross from the left by Holmes was bundled into his own net by the unfortunate Thomas.
- Sports & Recreation
- Craig Disley
- Martin Paterson
- Oliver Norwood
- James McKeown
- Danny Ward
- Ross Hannah