Sherwood has cause to be optimistic, though, as Tottenham have collected 29 points on their travels this season and boast the best away record in the Premier League.
"What's happened in the past we have to put to the back of our mind, we have to go there and impose ourselves and hopefully we can get a positive result," the 45-year-old Sherwood told a news conference.
"There is absolutely no pressure on the boys, our record is so bad we can go there and take the shackles off and enjoy the day.
"The motivation of the players will be there, it's a chance for them to put themselves in the record books.
"We are confident that we can spring a surprise and it is going to be a surprise if we beat Chelsea at the Bridge."
With Manchester City and Arsenal in FA Cup action this weekend, fifth-placed Tottenham could move to a point behind City and three adrift of arch-rivals Arsenal, who they host next week, with a rare victory over Chelsea.
But that will be no easy feat as Chelsea have dropped only four home points in the league this season, winning 12 games and drawing two, as manager Jose Mourinho remains unbeaten in 74 league games at Stamford Bridge in his two spells in charge at the club.
"We are going to respect them," Sherwood added. "They have got world-class players in certain areas of the field and we need to make sure we are compact enough to stop them playing first and foremost.
"He (Mourinho) has got a fantastic record since he has been in England and one that everyone needs to aspire to.
"If you can entertain and win at the same time then good, but I think the more important thing from Mourinho is to get the three points and invariably he does that."
Defenders Kyle Walker and Younes Kaboul face late fitness tests ahead of the trip to west London, but Christian Eriksen, Danny Rose, Etienne Coupe and Erik Lamela are all ruled out through injury.
Roberto Soldado, who scored his first goal at White Hart Lane from open play in the 1-0 win over Cardiff City last weekend, may continue upfront and Sherwood was delighted the Spaniard notched his sixth league goal.
"He must feel a lot better," added Sherwood. "Everyone is happy for him, he has always trained well and he has continued to do that this week.
"It was good for him to score and it was good for the reaction of the players, but I didn't expect anything different."
- Sports & Recreation
- Jose Mourinho
- White Hart Lane
- Gary Lineker