England play all three of their group games in Ukraine, where far-right hooligans were recently exposed by a Sky Sports News investigation.
Theo Walcott's family have already decided against travelling to watch their son play in the tournament over the fears.
A spokesperson for the Foreign Office told the Independent: "You cannot rule out the possibility of racism towards visiting fans.
"There is a risk in Poland and Ukraine as with many other countries where England play. We encourage visiting fans to report any incidents to the police."
Earlier this week assistant chief constable Andy Holt, who is heading the British police delegation at Euro 2012, admitted there could be dangers.
"I think football hooliganism is more of a problem in the Ukraine than perhaps it is at home," he said. "I don't think they are as advanced in dealing with racism."
New England manager Roy Hodgson conceded he was concerned about what might await England fans in the country.
"There's no doubt that the issue of racism, and the Sky report into hooliganism, and the violence in the Ukraine is a concern to us all.
"Not least the supporters who are going to go and maybe risk getting beaten up if they don't happen to be white."
Walcott's brother Ashley confirmed her family would not attend on Twitter: "Unfortunately my dad and I have taken the decision not to travel to the Ukraine because of the fear of possible racists attacks [sic]," he wrote.
"Some things aren't worth taking the risk, but begs the question why hold a competition of this magnitude in a place that cannot police itself."