Skipper Lee Carey's second-half penalty looked like being enough for Hastings as they led until the final minute of normal time.
But Tom Platt, who scored Harrogate's goal in the 1-1 draw the first time the teams met, was on hand at the back post to take the game into extra-time before Okojie's heroics in the shoot-out.
The Isthmian Premier side - the lowest-ranked team left in the competition - looked overwhelmed at the Pilot Field in the first half and were fortunate to still be level at the break after Luke Dean and Platt both went close.
However, just two minutes after half-time they took the lead when Zac Attwood was fould by Matt Bloomer in the box and Carey's placed effort put them in front from the spot.
Platt's equaliser came in the 90th minute as Harrogate looked to have run out of ideas, making the most of Matt Armstrong-Ford's failure to deal with a corner to poke home.
After a cagey period of extra-time Platt missed Harrogate's second penalty, blasting over the bar, but Danny Ellis did the same when given the chance to win it for the Arrows.
Lee Elam then proceeded to fire over with the first penalty of sudden death and Okojie coolly found the right-hand corner to seal a place in history for the hosts.
- Sports & Recreation