Watford arrived to the home of the champions in fine form, having beaten Arsenal 2-1 last weekend and not having lost on the road this season.
Despite going 1-0 down to Pedro's stunning 12th-minute goal, the visitors hauled themselves back into the contest in fine form by scoring twice either side of the break before they lost Roberto Pereyra to injury.
Antonio Conte's subsequent substitutions - Michy Batshuayi for Alvaro Morata, Willian for Marcos Alonso and Davide Zappacosta for Pedro - paid off as Chelsea scored the three goals they needed to secure a 4-2 win.
But Silva wasn't happy with the end result and thought Richarlison's two chances in the second half - which he missed - would have changed the game for the Hornets.
READ MORE: Chelsea Watford - full match report
READ MORE: Chelsea v Watford - how the match unfolded
READ MORE: Conte will ’never’ fear the Chelsea sack
"It’s a really unfair result," Silva told reporters at Stamford Bridge. "I think we didn’t deserve the result even if we took only one point I think it is not fair to us. It is football.
"We played more in the second half but even how we played first half we started very well. Of course, Chelsea scored a fantastic goal but we reacted well when we got to 1-1 we deserved that goal.
"After the first 25 minutes in the second half we played really good, we controlled the match. We didn’t give chances for Chelsea to counter. We scored the second goal after a fantastic chance for Richarlison and after the goal, [we had] two or three chances and we need to score to kill the match.
"A third goal would have killed the match and given us more chance to control in a different way. When you start to lose big chances to score it’s normal Chelsea with fantastic players start to believe. Then the fresh players in the pitch like Willian, Batshuayi - in one moment they can make the difference. It was a really unfair result."
Pedro's opener for Chelsea came from a corner, which should have been a goal-kick for Watford.
And Silva added: "Of course everyone knows it was not a corner. Everyone knows it is a goal-kick not a corner.
"Even in the second half I felt after the second goal all the corners and throw-ins were too easy, they fell to the offensive team in every moment.
"That’s football, it’s not fair. I don’t want to lose my time talking about these situations."