Running high on enthusiasm despite their 3-0 loss in Game One, the Devils clawed their way back from 2-1 behind with two unanswered goals.
The Madison Square Garden crowd left disappointed, but a series that promised to be a National Hockey League classic because of the team's bitter rivalry was starting to live up to expectations.
"Our team has a resilience to it and we usually respond in the right way, New Jersey head coach Peter DeBoer said.
"Every time we've been pushed and had our back against the wall in these play-offs we've come out swinging."
The Devils snatched the early lead when they finally found a way past New York's inspirational Swedish goalie Henrik Lundqvist as Russian left winger Ilya Kovalchuk unleashed a ferocious shot on a power play that whistled into the net after less than seven minutes of the opening period.
But the Rangers, who finished the regular season as the top ranked team in the conference, reasserted their control and dominated the middle period.
Defenceman Marc Staal tied the scores at 1-1, then center Chris Kreider put the home team ahead when he slipped the puck past Devils' netminder Martin Brodeur on another power play.
Ryan Carter scored for the Devils less than two minutes before the end of the period to tie the game at 2-2 before David Clarkson scored the go-ahead goal early in the third as the Devils, who took the day off training after their deflating Game One loss, finished strongly against their weary opponents.
"We've been in this position before. We just have to go into Jersey and try to get that next game," said New York captain Ryan Callahan.
"We have to get more shots. We definitely didn't get enough shots on net and we didn't have enough time in their end."
The series moves across the Hudson River to New Jersey for Game Three on Saturday. The series winner will play either the Los Angeles Kings or the Phoenix Coyotes in the Stanley Cup Finals.
"We have to care of business at home," said DeBoer. "This is a tough building to win at and we're glad to get a split but there's a lot of hockey left."