Former soldier Middleton, the host of military-based TV contest SAS: Who Dares Wins, was dropped by Channel 4 because of his “personal conduct”.
A statement from the broadcaster said it had “become clear that our views and values are not aligned”.
It comes after Middleton apologised following two separate social media incidents - one where he appeared to call activists “scum” and another where he said people following coronavirus safety measures were “sheep.”
However, Middleton, 40, suggested online he had stepped away from the show - adding he was“really excited about the future and what’s coming this year."
Middleton, who was on the show for five years from 2015, caused controversy in June following the Black Lives Matter protests - which took place across the world after the death of George Floyd.
At a time when there was nation-wide protests, and tension between activists and far-right groups, Middleton tweeted: “The extreme left against the extreme right. When did two wrongs make a right … BLM and EDL are not welcome on our streets, absolute scum”.
He later apologised and removed the Tweet. He said he was “anti-racist and anti-violence” adding he had not meant to describe Black Lives Matters protesters as “scum.”
Last year, Middleton said he would not be allowing Coronavirus to “dictate my life” - suggesting people who followed government guidelines and rules were “sheep.”
He later said he had not been fully aware of the developing situation in the UK as he had been filming in New Zealand.
The full statement from Channel 4 said: “Ant Middleton will not be taking part in future series of SAS: Who Dares Wins. Following a number of discussions Channel 4 and Minnow Films have had with him in relation to his personal conduct it has become clear that our views and values are not aligned and we will not be working with him again."
Middleton, a father-of-five, tweeted: “After 5 incredible years I’ve decided it’s time to move on from SAS Who Dares Wins UK. Big respect to my fellow DS – it’s been a journey I’ll never forget. Thanks to everyone that took part and made the show what it is. Really excited about the future and what’s coming this year.”
He added: “Stay positive.”
The show has spawned a number of celebrity specials in its five years - with reality star Joey Essex, model Katie Price and Olympian Victoria Pendleton amongst participants.
He also fronted Sky series Ant Middleton & Rebel Wilson: Straight Talking, which saw him go on a roadtrip with Australian comedic actor Wilson.