Joshua Zirkzee had heard his name chanted to the Emirates Stadium rafters by the travelling Manchester United fans when he was introduced as a substitute in the 81st minute. There has been a concerted attempt by them to build a few bridges after what happened in the Newcastle defeat at the end of December: ­Zirkzee applauded off after his 33rd minute withdrawal, a shattering low.

When the full-time whistle went here, this FA Cup third round tie locked at 1-1, Zirkzee turned towards the United support and geed them up. It felt as though the connection had been restored. The same supporters had got behind him when he came on in last Sunday’s 2-2 draw at Liverpool. And it just had to be Zirkzee who stepped forward at the end of the shootout with the kick to win it.

Related: Arsenal 1-1 Manchester United (3-5 pens): FA Cup third round – live reaction

The second half of normal time had flashed by, Bruno Fernandes scoring for United, Gabriel Magalhães equalising for Arsenal after the visitors had seen Diogo Dalot sent off in the 62nd minute. Arsenal blew the chance for 2-1 when their captain, Martin Ødegaard, had a hotly contested penalty saved by Altay Bayindir, whom Ruben Amorim preferred to André Onana. Bayindir would also make a couple of fine saves at the end of regulation time.

What a call it was by Amorim to give Zirkzee the fifth penalty. ­Everybody had scored apart from Kai Havertz for Arsenal, the German’s kick lacking power, Bayindir springing low to turn it away. If it set the seal on a horror day for Havertz, it burnished one for Bayindir that he will never forget – especially after the woes of his previous performance in the Carabao Cup quarter-final exit at Tottenham.

Now for Zirkzee and a moment of extreme redemptive possibility, the narrative arc impossibly perfect. When he buried his kick into the corner, the United fans would sing his name again. With even more feeling. United seemed to trade exclusively in difficult-to-script drama on their way to winning this competition last season. They are up and running in a similar vein.

For Arsenal, it was another body blow after last Tuesday’s 2-0 home loss against Newcastle in the Carabao Cup semi-final first-leg – and in a week that could be pivotal in their Premier League title push. They face Spurs on Wednesday and Aston Villa on Saturday.

Mikel Arteta and his players departed with regrets as they had the chances to have punished United’s 10 men. Profligacy was their enemy, Havertz a major culprit, the substitute Declan Rice another. To compound their frustration, Gabriel Jesus was taken away on a stretcher in the first half. Arsenal’s attacking options were already reduced, Bukayo Saka the headline casualty. Can the club afford not to make a January signing for the front line?

United have had no problems in the very biggest matches under Amorim; they have lacked the needed focus against the smaller fry. It was ­Fernandes who lit the touchpaper on 52 minutes after a virtual nonevent of a first half, taking a low cross from Alejandro Garnacho and shaping a lovely first-time shot into the far top corner.

The gauntlet was laid down to Arsenal, United reinforcing it with their physicality. Yellow cards fluttered in their direction. Dalot had got his first for a foul on Myles Lewis-Skelly and he was playing with fire when he leapt into a 50-50 with Mikel Merino. He got nothing of the ball and, when Merino took flight, it was a fait accompli for the referee, Andrew Madley, who had a very ­difficult afternoon.

United were upset with that decision. There would be worse to come for them. Havertz had sliced wide when well placed in the 54th minute and Arsenal turned the screw against the 10 men, Gabriel lashing home an equaliser on the spin after Bayindir could not get distance on a clearing punch, though Gabriel was aided by a deflection off Matthijs de Ligt.

Then came the controversy. Harry Maguire, a tower of strength at the back for United, looked to have stood his ground against Havertz, putting out an arm at worst. Havertz, though, went into it and tumbled over. United were incandescent when Madley pointed to the spot.

• Download the Guardian app from the iOS App Store on iPhone or the Google Play store on Android by searching for 'The Guardian'.

• If you already have the Guardian app, make sure you’re on the most recent version.

• In the Guardian app, tap the Menu button at the bottom right, then go to Settings (the gear icon), then Notifications.

• Turn on sport notifications.

A melee broke out, fury everywhere; a few head-to-head confrontations, too, including Manuel Ugarte against Havertz, who went down again. Ødegaard was forced to wait to take the penalty and Bayindir read him, plunging low to his left.

It was a fabulous save and Bayindir was not finished. He tipped over a header from Rice, who was gloriously placed and, at the end of normal time, he got something to another Rice effort, this one from an angle on the left. In between times, Havertz had lifted high from point-blank range after getting in behind Fernandes; he looked to have got away with a handball in the process. No video assistant referee, remember. Maybe Madley could have benefitted from the help.

Arteta had recalled Jesus on the right wing only to lose him after pretty much the only noteworthy incident of the first half. On came Raheem Sterling, who struggled sorely. Jesus appeared to catch Fernandes from behind as the United captain moved to shoot on the edge of the area – no foul, said Madley; he booked Fernandes for his furious reaction. Jesus twisted his knee badly in making the challenge.

Amorim stuck with his back three after Dalot’s dismissal, which was more like a five with the wing-backs holding tight and he mixed and matched as the game wore on, finishing with Lisandro Martínez in the centre-forward position.

Arsenal thought they had scored in the first period of extra time when Havertz crossed for the substitute Leandro Trossard only for De Ligt to make a clearance from under his own crossbar. United flickered in the final period through Zirkzee and another substitute, Amad Diallo. The shootout belonged to them.