Television replays showed the Spaniard digging his fingers into Vertonghen's face after the two came together following Torres's trip on the Belgian which earned him a yellow card during Saturday's draw at White Hart Lane.
"It's extremely difficult to know where to start," Villas-Boas said ahead of his side's Europa League clash with Russian side Anzhi Makhachkala in Moscow.
"Obviously, it is almost a farcical decision. It doesn't matter to me which player or club is involved."
Torres was later sent off for a second yellow card following another clash with the defender and the FA said on Tuesday that as a match official saw the first "coming together of the two players, albeit not in its entirety", it could not take retrospective action.
The striker would have faced a further three-game suspension if the FA had decided to charge Torres with violent conduct.
"Neither do I want to with my words put into question Fernando's integrity; neither their manager's, in trying to defend what is our position," added Villas-Boas.
"I think the FA has made a decision almost a joke. It looks incredible. How can you see the images, pretty clear, and come out without punishment on something which overtakes all professional behaviour?
"I think the decision is a disgrace. If the committee can't solve such clear images I don't think it sets out a good first example.
"I think the FA has lost all opportunity to put some sense into the images everybody saw."
Torres was given a one-match ban for the sending-off but would probably have missed Chelsea's game at Norwich City on Sunday anyway after picking up a knee injury in the Champion's League win over Steaua Bucharest on Tuesday.
- Sports & Recreation
- Anzhi Makhachkala