Rooney has failed to score in his previous two World Cup tournaments and underwhelming displays in England's warm-up matches have put his selection under the spotlight.
"I just think it is a witch-hunt we don't need," Neville told BBC Sport.
"Everything has been going so well with the team's preparation that people are looking for a negative now. But there isn't a negative with Wayne."
Neville continued: "There is a side of the English mentality to say 'let's look for a scapegoat and somebody to criticise'.
"But Wayne is our best player and that's a fact."
Rooney scored seven times in England's qualification campaign and netted 17 goals for Manchester United in the Barclays Premier League last season.
He will be hoping to carry that impressive form into England's first Group D match against Italy in Manaus on Saturday.
"Wayne's record in qualifying was superb," Neville said.
"He is held in high esteem throughout the world and is the player that, when his name is on the England team sheet, the Italian defenders will be thinking, 'he is the one we have got to stop'.
"Yet there is talk that we as a nation want to leave him out of our team. I cannot get my head round it.
"There is no doubt in my mind that Wayne is the first name on the teamsheet. Let's just hope that he delivers and these silly rumours and suggestions go away."
- Sports & Recreation
- Manchester United
- Paul Scholes