Seven-times Bulgarian champion Borisov, who was hoping to compete in the 69 kilogram category in London, was rushed to hospital on Sunday with a broken left wrist among his injuries after the fight in the western town of Bobov Dol.
Borisov was set to take part in the Bulgarian national team's training camps in Poland and Turkey as part of the preparations for the Olympic qualification tournament.
His coach was disappointed at not being able to count on Borisov for the Games.
"I feel sorry for the incident," Mihail Takov, a former world and European championship medallist, told Reuters.
"Evgeni is a tough guy and the bad thing is that he'll miss the Olympics. But we'll fight tooth and nail for each quota as we want to get Bulgarian boxing back where it belongs."
Police said in a statement that a verbal confrontation flared into an all out brawl involving 14 people, with another three taken to hospital.
Borisov, who also suffered a concussion and injuries to his right hand, told Bulgarian media on Wednesday he was frustrated to miss out on his chance for London.
"They're five against me and they hit me with a paving stone and a car jack," the 23-year-old said. "I have no explanation where this came from ... I'm so angry because this is the end of my Olympic dreams."
Bulgarian boxers have won four gold, five silver and eight bronze medals at the Olympics but came away empty handed at the 2008 Games in Beijing.