The Italian told Italian news agency ANSA in 2005 "I am a fascist, not a racist", and his appointment on Sunday to succeed Martin O'Neill prompted former foreign secretary David Milliband to stand down as Sunderland's vice-chairman and non-executive director.
But in a statement released by the club on Monday, Di Canio said: "I don't have a problem with anyone. I don't know why I have to keep repeating my story, to be defending myself on something that doesn't belong to me every time I change clubs. Talk about racism? That is absolutely stupid, stupid and ridiculous."
Miliband said on Sunday on www.davidmiliband.net: "In the light of the new manager's past political statements, I think it right to step down."
But former Swindon chairman Jeremy Wray, who gave Di Canio his first chance in management, dismissed that stance as a "sad knee-jerk reaction".
And Di Canio said: "What I can say is that if someone is hurt, I am sorry. But this didn't come from me - it came from a big story that people put out in a different way to what it was.
"The people who know me can change that idea quickly. When I was in England my best friends were Trevor Sinclair and Chris Powell, the Charlton manager - they can tell you everything about my character.
"I don't want to talk about politics because it's not my area. We are not in the Houses of Parliament, we are in a football club. I want to talk about sport.
"I want to talk about football, my players, the board and the fans. I don't want to talk any more about politics - I am not a politics person."
- Sports & Recreation
- Di Canio
- David Milliband