It is the fifth draw from eight games for O'Neill while Northern Ireland remain without a win since beating the Faroe Islands in August 2011.
Gareth McAuley did have the ball in the net in the 18th minute but it was ruled out for pushing and the visitors were grateful to goalkeeper Alan Mannus for two very good saves to deny former Coventry striker Michael Mifsud.
O'Neill selected his team with next month's World Cup qualifying match against Russia in mind.
Chris Baird and Kyle Lafferty are both suspended for that game and did not travel to Malta, so there was a Northern Ireland debut for Alex Bruce in the holding midfield role while Walsall's Will Grigg won his second cap up front.
Bruce, who plays under his father Steve at Hull, previously played for the Republic of Ireland but was able to switch allegiance as he did not make a competitive appearance.
St Johnstone's Mannus started in goal with regular keeper Roy Carroll absent through illness while Mifsud captained Malta.
O'Neill has fond memories of Ta'qali stadium having scored his first international goal here in 1989, while goalkeeping coach Tommy Wright made his debut in the same game and assistant manager Billy McKinlay also scored his first goal for Scotland in Malta.
It was the hosts who made the brighter start, though, and Northern Ireland were almost behind in the fifth minute when Andrew Cohen crossed for Andre Schembri, who should have scored but shot straight at Mannus from eight yards.
Grigg fought off competition from Scottish Premier League top scorer Billy McKay among others for the striking role, but he fluffed his lines when his first big chance came along in the 17th minute, a heavy touch allowing keeper Andrew Hogg to block.
Moments later Northern Ireland had the ball in the net when McAuley headed in a Chris Brunt corner but Bulgarian referee Nikolay Yordanov disallowed it for pushing.
Mifsud was proving a handful for the visiting defence and his trickery created another opening for Schembri, whose drive beat Mannus but flew just past the angle of post and bar.
Malta were looking the more likely to break the deadlock and Clayton Failla was the next to threaten Mannus' goal with a fierce free-kick from 30 yards that was a yard over the bar.
Wingers Niall McGinn and Shane Ferguson were having little joy down the flanks but in the 36th minute McGinn made progress down the right and Steven Davis crossed for Grigg to glance a header wide.
O'Neill made one change at half-time, Craig Cathcart replacing Jonny Evans in the centre of defence.
The Ta'qali has traditionally been a good venue for Northern Ireland, with 2005's 1-1 draw between the sides the only one of six previous meetings they did not win.
O'Neill's side have struggled to score on the road, though, and remarkably were looking for their first away goal in a friendly since beating Finland 2-1 in 2006.
The visitors almost caught Malta cold as McGinn broke through in the middle but Hogg was off his line quickly to punch clear, while Aaron Hughes then did well to get to the byline on the right and pull the ball back but McGinn scuffed his shot.
In the 56th minute Mannus was called into serious action for the first time when Mifsud unleashed a stinging shot from the right, the keeper palming it away.
In the 63rd minute Inverness striker McKay was introduced, winning his first cap when he replaced Ferguson on the right, with Grigg remaining the lone striker.
The visitors were pressing but in the 70th minute they were indebted to Mannus for an excellent stop to keep out Mifsud once again.
Pat McCourt and Josh Magennis were introduced late on but there was no way through the Malta defence and the visitors had to settle for a disappointing draw.
- Sports & Recreation
- Alan Mannus
- Northern Ireland
- Michael Mifsud