Yet none has gone on to change his sport like Cassius Clay, who won light-heavyweight gold in Rome in 1960.
The journey to Rome was problematic in itself for the 18-year-old from Louisville, whose fear of flying meant he tried to pull out of the Games - he was eventually persuaded to fly, but not before purchasing a parachute in case of emergency.
Though considered a promising youngster, few thought Clay was destined for stardom, though the man himself's brashness and confidence earned him the nickname "the mayor of the Olympic Village".
In the ring Clay was next to flawless, stopping his first opponent, Yvon Becot, in the second round, before winning unanimous verdicts in his remaining bouts - outclassing Zbigniew Pietrzykowski to claim gold.
Few will need reminding what happened next. He beat Sonny Liston to become world champion in 1964 and never looked back. As Muhammad Ali, he established himself as one of the most important sporting and cultural figures of the 20th century.
Ali later wrote that he threw his gold medal into the Ohio River after being refused service in a whites-only restaurant.