Jon Daly's last minute penalty came after the Tangerines had raced into a 3-0 lead inside eight minutes - before Billy McKay's hat-trick followed by Gary Warren's 86th-minute header looked like earning the points for Inverness.
United went ahead through Josh Meekings' early own goal before further strikes from Daly and Rudi Skacel seemed to have them home and dry, until McKay sparked a comeback which few inside Tannadice would have predicted.
Houston said: "It feels more like a defeat if I'm being honest. When you go 3-0 up so early on in any game and score goals at home you really do expect to win the match
"The only thing I'm pleased about is the fact we got a late penalty to get us a point.
"We didn't defend crosses into the box all that well which is something we've been working hard on recently.
"We're all naturally frustrated with the end result. You would expect that from an honest bunch of boys in the dressing-room.
"At their fourth goal we had a chance to clear it but didn't stop the cross and were punished because of that.
"When it did come into the box, there were enough Tangerine jerseys to get their head on it but they were more hungry to get there, which was disappointing.
"Inverness will go home the more happier of the two teams, which is understandable."
- Sports & Recreation