In his last ever game for the Welsh region, the diminutive winger ducked out of Rob Kearney's challenge to wriggle over in the right corner for a 78th-minute try.
Williams' score made it a one-point game and Dan Biggar showed nerves of steel to land the difficult conversion, etching the Ospreys into the record books as the first team to win four league titles.
The late knockout punch left the RDS crowd stunned as Leinster were on course to complete a famous European and domestic double - a feat not achieved since London Wasps did so back in 2004.
Joe Schmidt's men had led 17-9 at half-time thanks to tries from Sean Cronin and Isa Nacewa, and another from the Fijian international gave them a 30-21 advantage.
However, the Ospreys showed tremendous grit and will to win as Biggar landed his fourth penalty and then added the extras to Williams' effort to seal a famous comeback win.
The defeat is a bitter pill for Leinster to swallow as it is their third straight loss in a league decider, the Ospreys repeating their 2010 victory in Dublin.
Steve Tandy's side had beaten the Heineken Cup champions twice during the regular season and they nipped ahead thanks to a ninth-minute penalty from Biggar.
Leinster were level within two minutes though, Jonathan Sexton delivering a crisp penalty strike to reward the carrying of Gordon D'Arcy and Mike Ross.
Prop Ross injured his left leg at a subsequent scrum - a concern ahead of Ireland's summer tour to New Zealand - and Kiwi Nathan White had to be introduced.
Leinster had the better of the possession in a fiercely contested first quarter, but pressure from the visitors' blitz defence forced a turnover and Andrew Bishop broke downfield.
He linked with Rhys Webb who was hauled down in sight of the posts as Leinster scrambled back, with the Ospreys having to settle for a Biggar three-pointer.
But the Ospreys struggled to use their scrum as a weapon and the Irish province charged back into scoring range, retaining possession through seven phases.
That set up an opportunity on the left for Brian O'Driscoll to brilliantly draw in two defenders and set hooker Cronin free for a rampaging run to the line.
Sexton converted with aplomb, a quality kick which was matched by the in-form Biggar as he topped off a powerful scrum with his third successful penalty.
However, Sexton's pinpoint restart kick was fielded superbly by a leaping Nacewa and the winger burst through for an opportunist try.
That defensive lapse was further punished by Sexton adding the extras from wide out, although the Ospreys quickly got back on the front foot.
There were extraordinary scenes as Leinster were camped close to their own line for eight minutes before half-time, frantically defending scrum after scrum.
They had loosehead Heinke van der Merwe sin-binned for dropping his bind against Adam Jones. But youngster Jack McGrath dug in and eventually, with the Ospreys failing to get the shove on and Jones being singled out, referee Romain Poite awarded a penalty to the hosts.
The Ospreys felt aggrieved and as the players headed for the dressing rooms, there was an altercation between the forwards with Alun-Wyn Jones and Jamie Heaslip having to be separated.
Suitably fired up, the Welshmen had the try they craved just 84 seconds into the second half. Joe Bearman's clever offload released centre Ashley Beck for a well-taken seven-pointer.
Leinster replied almost immediately, Sexton rifling over his second penalty and van der Merwe and Kevin McLaughlin were back on the pitch just moments later.
Ospreys captain Jones infringed at a ruck to allow Sexton to kick Leinster 23-16 ahead and Nacewa then went close on the left after Shane Jennings scooped up a turnover.
In yet another twist, the Ospreys worked numbers on the left to release Williams for a rare scoring chance and he showed his renowned finishing skills by crashing over past Eoin Reddan.
Biggar missed the conversion, leaving Leinster with a two-point buffer, and the league's table-toppers seemed to be turning the screw when Nacewa swooped for his second of this closely-fought final.
A surging maul got Leinster in sight of the whitewash and Nacewa was able to collect Reddan's loose pass and stretch over from close range.
Sexton's conversion completed his 15-point haul and pushed the margin out to nine points with just 16 minutes remaining.
It was an uphill task for the Ospreys but with replacement Kahn Fotuali'i getting them moving well, Leinster were suddenly pinned back and starved of possession.
The defensive workload took its toll on the home side, who had prop White sin-binned for successive scrum infringements. That led to uncontested scrums, and Biggar sent over the resulting penalty to give his team-mates renewed hope.
With their forwards making the hard yards, the Ospreys went for the jugular and as they flooded out to the right, ace poacher Williams had enough of a sniff to go for the kill.
The 35-year-old had scored better tries in his regional career but few have been more important. He passed the baton to Biggar, possibly one of Welsh rugby's future stars, to loft over the match-winning kick.