The controversial batsman's England career was ended in February when he was left out of World Twenty20 in Bangladesh and the preceding tour of the West Indies.
Downton, who was instrumental in the decision to end Pietersen's international career, said the batsman had become "disinterested" on the pitch and that Australian captain Michael Clarke had "played him like a schoolboy".
"There was an enormous amount of frustration surrounding KP from everyone in the management team in terms of his attitude and whether he was really fighting it out," Downton told BBC's Test Match Special programme.
"There was a feeling KP wasn't engaged in the way he should be. I talked to quite a few senior players and couldn't find one supporter who wanted KP to stay in the side."
Downton said he was shocked at Pietersen's performance in the fifth test in Sydney which England lost by 281 runs with Pietersen contributing only nine runs.
"I watched every ball and I've never seen anyone as disinterested or distracted on a cricket field as Kevin," he said. "It led me to talk to every person on the management team within England and a lot outside it.
"I then spent the next three weeks speaking to more people but the decision we came to was in the best interests of English cricket. I'm not saying all that happened in Australia was down to KP, team dynamics disintegrate when you're under pressure."
Pietersen played 104 test matches for England, averaging 47, but managed only two half centuries in the five tests in Australia. Downton said Pietersen's refusal to dig in for the team and a series of soft dismissals hastened the end.
"It was frustrating seeing him with his 'this is the way I play' type attitude'", Downton said. "Clarke played him like a schoolboy almost."
England's Ashes debacle also led to coach Andy Flower leaving along with batting coach Graham Gooch while spinner Graeme Swann retired before the end of the series.
- Sports & Recreation