Hilare Momi scored twice and David Manga grabbed the winner as the visitors came from 2-1 down - and the set-back of having goalkeeper Lembet sent off in the first half - to win the first round, first leg tie 3-2 in Alexandria.
Egypt had never before lost at home in Nations Cup qualifying, a sequence of 38 matches stretching back to 1965.
Playing behind closed doors because of security concerns on the eve of presidential elections, the hosts took a 10th-minute lead through Mohamed Zidan but Momi equalised in the 26th.
Defender Salif Keita was sent off before Mohamed Salah netted just before the break for Egypt.
The Central Africans, whose coach quit last month because he had not been paid for months, then bounced back with two second-half goals.
Egypt are now in danger of missing out on the finals for the second tournament in a row after having won an unprecedented three back to back Nations Cup titles between 2006 and 2010.
They must win the second leg in Bangui on June 30 by two goals to progress.