Southern League Premier Division side Stourbridge pulled off an FA Cup upset over 10-man Plymouth to book their place in the second round for the first time in their history.
Second-half strikes from Paul McCone and Sean Evans sealed a famous replay win for the Glassboys over npower League Two's bottom side, who had Paul Bignot sent off before the break.
McCone's 52nd-minute header put the home side in front and Evans sealed Stourbridge's passage to a second-round home tie against Stevenage when he bundled home a 73rd-minute rebound.
Grimsby were also celebrating a giantkilling of sorts after the former Football League club - now in the Blue Square Premier - beat League Two Port Vale 1-0 after Serge Makofo's 59th-minute strike.
On a night of upsets, AFC Wimbledon were one of three League Two sides to overturn higher-ranked opposition with Luke Moore's 66th-minute goal giving them a 1-0 win over Scunthorpe at Glanford Park.
Liam Dickinson volleyed a 55th-minute winner as Southend saw off Preston by the same score at Roots Hall, and second-half strikes from Garry Richards and Stefan Payne helped Gillingham hang on for a 3-2 home win over Bournemouth.
Blue Square Premier leaders Wrexham reached the second round with goals from Mathias Pogba and Danny Wright nudging them to a 2-1 win over Cambridge, who replied late through Michael Wylde.
And Blue Square Premier Tamworth booked a second-round trip to Gateshead after a dramatic 1-0 win over Hinckley, Kieron St Aimie bundling home a Liam McDonald corner in the dying moments.
Aldershot will travel to Sheffield Wednesday in round two after goals from Alex Rodman and Luke Guttridge gave them a 2-0 win over Maidenhead of the Blue Square South.
Ben Bradbury's extra-time winner ensured Isthmian Division One North side Redbridge - the lowest ranked side left in the competition - reached round two with a 2-1 replay win at Oxford City.