Cavendish, who won the green jersey for the points classification last year and at 26 already has 20 Tour stage wins to his name, would not enter the race with the goal of defending the shirt.
"Mark has said he wants to finish the Tour, but if he knows he can win the Olympics he will take the final week day by day," coach Rod Ellingworth, who is also Britain's Olympic road-race team manager, told The Guardian online.
"If the green jersey comes, it comes. He wants to win stages at the Tour, not have the green jersey as his main objective. There is a chance he could get off in the Tour if he is tired and is digging himself into a hole."
The Tour ends in Paris on July 22, with the London Olympic road race scheduled for July 28.
Should Cavendish actually give up his green jersey ambitions, his Team Sky could favour compatriot Bradley Wiggins, who emerged as a potential Tour winner after his Paris-Nice victory this month.