The Devils delighted the packed Prudential Center crowd of 17,000 with goals from Zach Parise, Stephen Gionta and Patrik Elias within the first 6:16 of the game, but Florida refused to bend and used the powerplay to take the series lead.
Two other visiting NHL teams also had cause to celebrate as the Nashville Predators beat the Detroit Red Wings 3-1 to take a 3-1 lead in their series, with a chance to clinch at home when the series resumes on Friday.
Also, Mikkel Boedker scored the game-winning goal in overtime as the Phoenix Coyotes beat Chicago 3-2 to take a 2-1 series lead in a series that has featured three extra-period wins in as many games.
Florida fought their way back by scoring twice on the powerplay in the last four minutes of the opening period, drawing within 3-2 when Jason Garrison beat Devils goalie Martin Brodeur from just inside the blue line with eight seconds left.
The Panthers pounced again in the second period, with Mike Weaver scoring on a deflection past Brodeur after 2:18 to tie the game and send the NHL's record holder for goalie wins to the bench in favour of Johan Hedberg.
The switch did not slow the Panthers down as they made it 3-for-3 on the powerplay when Brian Campbell scored past Hedberg to give Florida a 4-3 lead.
"We've been burned in two games by taking penalties," Devils coach Peter DeBoer lamented. “"Their powerplay has been the difference."
The Panthers also scored two powerplay goals in their 4-2 win at home on Sunday.
Florida back-up goalie Scott Clemmensen, who replaced Jose Theodore after the three quick Devils' goals, ensured his team held the lead, stopping all 19 shots he faced.
"Down by three goals or up by three goals, we keep playing. We play to the buzzer regardless of what happens," Clemmensen said.
"The momentum swings in this series have been drastic. It teaches you a lot about unity and perseverance."
Brodeur said he was surprised to be removed from the game.
"I think we got a little over-excited about having a three-goal lead and got a little undisciplined," said Brodeur, who was last pulled from a game six years ago.
"You don't expect when you're not losing the game yet to get pulled."
New Jersey thought they had made it 4-4 in the waning moments of the second period off a shot by Marek Zidlicky, but the referee ruled that Devils right winger Steve Bernier had unintentionally interfered with Clemmensen.
Game Four will be played in New Jersey on Thursday.