In a pre-recorded video message to the Football Supporters' Federation summit at Wembley, former BBC director general Dyke, who took over from David Bernstein at the FA a year ago, said the governing body was in danger of becoming "irrelevant".
"If you look at who's supporting, who's playing, football and then you look at the FA Council - it doesn't represent them," Dyke was quoted as saying on the BBC.
"It's still overwhelmingly male, overwhelmingly white in a world that isn't overwhelmingly male and white and somehow that has to be changed.
"We have to try and change it but we're not alone, supporters have got to try and change it as well."
The FA Council consists of around 120 members from both the professional and amateur circles of the game and are responsible for helping to make policy decisions for the organisation.
However, it has only one supporter representative, Malcolm Clarke, who is also the chair of the FSF.
The comments of Dyke, who once accused the BBC of being "hideously white", come after criticism of the FA's lack of diversity by the likes of former England defender Sol Campbell.
Campbell said in his biography this year that, had he been white, he would have captained England for 10 years.
"We've got to look at how we involve the population of 21st century England and the mix it's got," said Dyke. "If we just carry on like this - old, white males - we're going to be increasingly irrelevant."
- Sports & Recreation