Andre Santos spurned two golden chances as Arsenal were unable to make their first-half dominance count.
Arsene Wenger's side were punished when Azmi Muslim beat Vito Mannone with a thunderous strike from 25 yards that flew into the top corner.
Wenger made 10 changes at the break and the newly-introduced youngster Benik Afobe impressed, striking the post after a quickly-taken free-kick.
With time running out Arsenal levelled through German Thomas Eisfeld, before Chuks Aneke won it in the last minute with a deflected effort.
Arsenal faced a composite Malaysia XI last year, with goals from Aaron Ramsey, Theo Walcott, Tomas Rosicky and Carlos Vela ensuring an easy victory for Wenger’s side.
But things were different this time around, as Wenger experimented with a 4-3-3, using full-back Santos alongside Gervinho and Marouane Chamakh in attack.
While much of the first half was encouraging for Arsenal, they struggled to finish the chances they created – with Santos the chief culprit as the Gunners misfired.
The Brazilian’s first sight of goal was after 30 minutes, missing the target by some distance when he should have slotted in.
But his miss a few minutes later was even worse. Seizing upon a loose Malaysia back pass, Santos found himself with the goal at his mercy – but could only hit his shot straight at Razali.
Malaysia punished Arsenal in emphatic fashion in first-half stoppage time as Muslim’s thunderbolt lifted the roof inside the stadium.
Arsenal came out strongly at the start of the second half and could have levelled when Afobe was sent clear down the left - but could only find the base of the left post.
Mohd Sali was the star for the hosts and he could have made it 2-0 when he got the better of Kyle Barkley, but could only find Mannone when he tried to squeeze the ball in at the near post.
But Arsenal looked dangerous whenever Alex Oxlade-Chamberlain and Ryo Miyaichi were given space to attack and the equaliser, while perhaps unexpected, was not undeserved.
Alex Song started the move with a pinpoint long ball to Walcott, who cut back for Eisfeld to convert from 10 yards.
But the Arsenal winner minutes later was cruel on the hosts, who acquitted themselves well against a fresher and more talented Arsenal. But it came courtesy of Aneke, who drove into space 20 yards from goal and saw his shot take a huge deflection and bounce into the net.
Arsenal will know they were fortunate to get the win and Wenger will demand better when they travel to Beijing to face Premier League champions Manchester City on Friday.