But Southampton’s final match under manager Ruben Selles, who is expected to be replaced by Swansea boss Russell Martin in the coming days, ended all square after Gakpo and Jota struck in the space of a minute.
But the Merseyside club had to be content with extending their unbeaten top-flight run to 11 games, having begun the day knowing they would miss out on a top-four finish for the first time since 2015-16.
With Southampton seeking to restore a modicum of pride after their fate was sealed a fortnight ago, Liverpool initially looked like they would canter to victory.
Dreadful defending gifted the visitors the 10th-minute opener as Jota fired into an unguarded net from close range after being teed up by a woeful pass from Romeo Lavia as Saints attempted to play out from the back.
Firmino swiftly doubled the Reds’ advantage, collecting a pass from Fabinho on the edge of the hosts’ 18-yard box before dummying his way beyond Lyanco and Jan Bednarek and driving through the legs of Saints goalkeeper McCarthy.
There was a strong sense of deja vu for long-suffering home fans who have witnessed just two home league wins all season but Southampton responded by showing the fight they have so often lacked.
Long-serving midfielder Ward-Prowse – who was potentially making his final Saints appearance ahead of a mooted summer move – halved the deficit in the 19th minute by coolly slotting into the bottom-right corner after being picked out by Carlos Alcaraz.
And Selles’ side were level just nine minutes later.
Firmino sloppily conceded possession to Lavia close to the halfway line as Liverpool attempted to break, allowing Theo Walcott to slide in Sulemana, who fired his first goal in English football under Kelleher.
Sulemana stylishly completed Southampton’s stunning comeback just two minutes into the second period.
The Ghana international collected the ball midway inside his own half, eased past Fabinho and then accelerated unchallenged to the edge of the box before bending into the bottom-right corner and celebrating with a backflip.
And the Reds were soon facing a major uphill battle to salvage something as substitute Armstrong made an immediate impact.
A minute after replacing Lavia, the striker intercepted Jordan Henderson’s careless pass and raced forward before his low-angled finish into the bottom-right corner seemed to catch Kelleher out of position.
Liverpool were stunned by the extraordinary turnaround but intent to protect an unbeaten run dating back to April 1.
Gakpo – a one-time Southampton target – halved Saints’ lead by tapping in Trent Alexander-Arnold’s volleyed cross in the 72nd minute before Jota found space to lash home his second from Salah’s pass moments later.
Salah almost snatched victory for the Reds 11 minutes from time but his attempted control from a long pass struck the left post after looping over the head of McCarthy and the spoils were shared.