The Egyptian-born businessman wants to rebrand the 109-year-old club 'Hull Tigers', a move that is deeply unpopular with supporters who have formed a protest group called City Till We Die.
"They can die as soon as they want, as long as they leave the club for the majority who just want to watch good football," Allam told the Independent on Sunday newspaper.
"How can you be supporting a club when you distract attention during a game?
"I'm a simple man. Do they want me to stay? If it's 'No thank you', fine, in 24 hours the club is for sale ... I do not put in one more pound."
The 74-year-old Allam arrived in Hull as a student in 1968, rescued the club from the brink of bankruptcy in 2010 and has invested around £35 million.
He changed the name of the club's holding company from Hull City Association Football Club to Hull City Tigers in August.
He believes a 'Tigers' brand would have more of a "global marketing impact" having previously said he considered the word 'City' to be "lousy" and "common".
"How can they call themselves fans, these hooligans, this militant minority, when they disturb and distract the players while taking away the rights of others to watch the football, and of companies who have paid good money for their advertising?," Allam said on Sunday.
"If they want to express their feelings they are free to do so, either outside the stadium or pay to take space.
"Seriously, they are welcome to talk to the stadium management about buying a space for a permanent banner, 10 times as big if they want. I am a supporter of democracy. I would have no issue with that," added Hull's owner.
- Sports & Recreation