Ferdinand has known nothing but success since he joined United 12 years ago, winning six Premier League titles and one Champions League.
The defender has, therefore, found life hard going this year because of the team's struggle under David Moyes.
United are seventh in the table, 18 points behind leaders Chelsea with seven matches remaining.
And Ferdinand concedes he cannot bear the idea of stepping out in public after every one of the team's embarrassing defeats.
"I don't want to go out when the game has finished," Ferdinand said in a frank interview with BT Sport.
"I don't want to take my kids to school and listen to things that I think I'm hearing or see people look at me in a different way because we've lost a game.
"I don't want to go out and fill my car up with petrol or go to the shops or anything. It's embarrassing.
"Ideally you want to play the next game the next night and get it out of the way so you can go out and get a win and go out with your chest out and feel more comfortable because it's personal.
"You think everyone is talking about you and the result and what's going on with the club and the form and you don't want to live like that.
"You want to walk freely and comfortably."
United's season sank to a new low on Tuesday when they were beaten 3- 0 by neighbours City in the Manchester derby.
Moyes' side take on Aston Villa on Saturday before hosting Bayern Munich in the Champions League quarter-finals on Tuesday.
Another big defeat in either fixture will heap further pressure on Moyes.
- Sports & Recreation
- David Moyes
- Premier League