The Big Apple is still in a frenzy of sporting excitement after the New York Giants won the Super Bowl in February and now the Rangers have given their long suffering fans real hope they can win the NHL title.
The Rangers caught the visitors napping when centre Brad Richards scored after just 92 seconds but then had to survive a relentless fight-back from the Capitals in the seventh and deciding game of their Eastern Conference quarter-final.
Led by their Russian captain Alexander Ovechkin, Washington launched a ferocious wave of attacks on New York's defence but the Rangers' Swedish goalie Henrik Lundqvist stood firm.
The Rangers extended their lead to 2-0 midway through the third and final period when defenceman Michael Del Zotto scored.
But the Capitals, who won a seven-game thriller over last season's Stanley Cup champions the Boston Bruins just to reach the second round, pulled one back 38 seconds later to set up a nail-biting final 10 minutes.
Czech Ramon Hamrlik buried the puck into the Rangers net after Lundqvist's view was blocked, but the Caps were unable to score again, losing the series 4-3.
The Rangers, who were also pushed to seven games by the Ottawa Senators in the opening round, reached the Eastern Conference final for the first time since 1997. They last won the Stanley Cup in 1994.
Their next opponents are their cross-river neighbours, the New Jersey Devils, who wrapped up their series against the Philadelphia Flyers on Tuesday, giving them an extra four days rest.
The teams will meet in game one at Madison Square Garden on Monday with the eventual winner to play either the Los Angeles Kings or the Phoenix Coyotes for the Stanley Cup.
The odds are seemingly stacked against the Rangers, who have struggled in the play-offs after they finished the regular season as the number one team in the conference, as no team has won the Stanley Cup after being pushed to seven games in the first two rounds of the post-season.