Hayward knocked over the winning kick from 25 metres after replacement back Maritino Nemani made a rash late tackle after the fullback had kicked clear.
It was just the Force's third win of the campaign and left the visiting Highlanders rooted to the bottom of the table after they failed to add to their sole success in 2013.
The Force enjoyed the bulk of possession and territory in the first period as once again the Highlanders' big names struggled to make an impact.
A snaking run by Hayward set up the game's only try in the 16th minute as he was finally brought down inches short of the tryline but quick ball was fed to flyhalf Sias Ebersohn who touched down in the corner for the opening score.
Hayward and Hayden Parker then exchanged penalties but the Force fullback missed a straightforward effort just before halftime to leave the home side ahead 10-6 at the interval.
Colin Slade came on as a second half replacement for the Highlanders and he kicked two penalties which were matched by Hayward to leave the Force 16-12 ahead on the hour mark.
Force scrumhalf Alby Mathewson was then sent to the sin bin for tackling the halfback at a ruck but the Highlanders could not find a way past the 14 men with some poor decision making and last gasp tackling preventing a try.
Slade did knock over a third penalty to cut the deficit to one point and the All Black flyhalf smashed over a lengthy 50 metre effort with three minutes remaining to put the visitors 18-16 ahead and in sight of victory.
But Nemani's error proved crucial with Hayward's fourth penalty claiming their first win in six matches.
- Sports & Recreation
- Maritino Nemani