Despite a squad budget of around five million euros (£4m) - the smallest in the league - minnows Paderborn refused to buckle against their more experienced rivals, rewarding their 14,000 fans with their first point in their maiden top-flight appearance.
Japan international Shinji Okazaki fired Mainz into the lead after tapping in from five metres following a free kick.
The North Rhine-Westphalian club, who finished a surprise second in the second division last season, kept their cool and earned an equaliser with Elias Kachunga's powerful drive from 17 metres.
Uwe Huenemeier's header three minutes from time from a Marvin Bakalorz cross triggered wild celebrations as the hosts thought they had sealed a memorable first win.
But Mainz kept at it, Okazaki rattling the post with a header in the last minute of regular time as they dug deep to avoid a season-opening defeat.
They were awarded a penalty in stoppage time after Huenemeier went from hero to villain in just a few minutes, bringing down Okazaki. Koo Ja-cheol converted the spot kick to save Mainz's blushes.
But it was fellow substitute Kramer, who had come on in the 74th minute, who snatched the last-gasp equaliser and a point for the hosts with a low drive from inside the box.
Champions Bayern Munich got off to a winning start on Friday, beating VfL Wolfsburg 2-1, while last season's runners-up Borussia Dortmund slumped to a 2-0 home defeat against Bayer Leverkusen on Saturday.
- Sports & Recreation