Tsitsipas won 6-1 4-6 7-6 (6) at the O2 Arena to see Dominic Thiem qualify for the last four, having beaten Nadal earlier in the day, and set up a winner-takes-all clash with the Spaniard on Thursday.
Tsitsipas was dejected after losing his opening match to Thiem but reacted in just the right manner on Tuesday night, wrapping up the first set after only 19 minutes.
However, Rublev, making his debut in the tournament, produced a spirited response and had a match point in the deciding tie-break only to double fault.
Tsitsipas said in his on-court interview: "It was an unbelievable match from both sides. We produced some incredible tennis. The relief at the very end of putting all that effort and all that fight on the court paid off.
"I'm very happy that I showed determination and a willingness to not give up when he had match point. I produced some really good tennis and that helped me take the win."
Russian Rublev qualified for the tournament for the first time by virtue of winning more titles and matches than anyone else this season.
Additional reporting by Press Association.