The tourists had reached 195 for the loss of only one wicket when Elgar, who hit 11 fours and three sixes in a knock of 103, was dismissed.
But they subsequently finished Wednesday's final session on 268 for five, with Dilruwan Perera and Suranga Lakmal claiming two wickets apiece.
Faf du Plessis contributed 80 to the South African effort, while Hashim Amla - in his first innings since being named as his country's new Test captain - could only manage 11 before being ousted.
- Sports & Recreation
- Suranga Lakmal
- Dilruwan Perera
- Hashim Amla