Burke's 74th-minute shot was defected past helpless keeper David De Gea by defender Axel Tuanzebe to secure a 2-1 win, just the Blades' second in the league this season.
Defender Kean Bryan had earlier headed them in front with De Gea appealing in vain that he had been impeded as he attempted to collect John Fleck's corner - and Anthony Martial was denied a goal at the other end as referee Bankes ruled that Harry Maguire had fouled Blades goalkeeper Aaron Ramsdale.
Maguire looked to have dashed his former club's hopes with 64 minutes gone when he powered home a header from Alex Telles' corner to level, but United were undone once again with 16 minutes remaining as substitute Burke's drive cannoned off Tuanzebe and flew past De Gea into the net.
The result saw United pass up the opportunity to retake top spot from rivals Man City, and raised serious questions about their mentality in what could be a nervy title race.
Solskjaer was unimpressed by the officiating, telling BT Sport after the game: "The first goal is a foul, Billy Sharp runs into David de Gea.
"At the other end, no foul for the disallowed goal for us but it's that kind of season. Two mistakes by the referee."
However, Solskjaer did not let his players off the hook: "We had all the possession but when you concede two bad goals it's always going to be difficult. We didn't create big enough chances to score enough goals.
"That magic was missing, that little bit extra. They defended well, fair play to them but we didn't have the right ideas or solutions.
"It wasn't to be, the second goal we conceded is so poor, so sloppy. Easy. We stopped getting out to the ball, three or four bad decisions which is out of character."
Next up for United is a trip to in-form Arsenal on Saturday, and Solskjaer says the Red Devils must move on quickly.
"There will be no big inquest," he added. "There have been so many odd results this season but with the world as it is we have been the most consistent team, it hit us today."
Watch: Level Headed with Leon McKenzie
Additional reporting by PA.