Roared on by a boisterous Madison Square Garden crowd, the Rangers gave their long suffering fans cause for optimism that an overdue NHL title could be close when Martin St. Louis scored the Game Four winner six minutes into overtime.
The result gives the Rangers a commanding 3-1 lead in the best-of-seven Eastern Conference finals with the next game in Montreal on Tuesday.
St. Louis sealed the win when he combined with Carl Hagelin and Brad Richards to slap the puck past Canadiens goaltender Dustin Tokarski from an acute angle, whipping the Big Apple into a frenzy of excitement after a tense game.
"It was a really competitive game but we put ourselves behind the eight ball with all those penalties," Rangers coach Alain Vigneault told reporters.
"It was us. They were killing us and we can't afford to do that."
Hagelin had opened the scoring just seven minutes into the first period with a short-handed goal when his team mate Benoit Pouliot was in the penalty box for high sticking.
The Canadiens have struggled with their power play during the series and their inability to score was compounded when Ryan McDonagh and Brian Boyle got the puck to Hagelin on the breakaway.
The Swedish left wing skated free then shot through the legs of Tokarski for his sixth goal of the playoffs. Tokarski, called into the starting lineup after Corey Price was ruled out of the series with a knee injury, made 26 saves.
The Canadiens tied the game at 1-1 midway through the second period when Montreal defenseman Francis Bouillon rifled a snap shot past the Rangers goalie Henrik Lundqvist, who turned away 27 shots.
"It's a big relief, an exciting at the same time, to get this win," said Lundqvist.
"We're not thinking to ourselves we're one game away. We have to keep the same mindset as every other games because it's going to be a tough game.
"But it's still exciting to know you're one game away from the finals."
New York regained the lead when Dan Girardi fed Derick Brassard and the Rangers center buried the puck past Tokarski on the last minute of the second period.
After killing off the first six Montreal power plays, the Rangers' luck ran out early in the third period when defenseman John Boyle was banished to the penalty box for tripping.
The Canadiens' finally took advantage of the extra man when PK Subban's slap shot took a deflection past Lundqvist to tie the game at 2-2 after two minutes of the final period.
Alex Galchenyuk almost won the game for the Canadiens with three minutes to go in regulation but his snap shot hit the crossbar off Lundvist's shoulder, forcing the teams into overtime for the second game in a row.
"We got a few chances," said Canadiens coach Michel Therrien. "We had a lot of opportunities on the power play but we couldn't take advantage of them.
"We did score one goal but we also gave one up and I thought that was probably the game."