The narrow victory sees the Buccaneers join SuperSport United, Lamontville Golden Arrows as well as Chippa United in the semi-finals.
Bucs head coach Kjell Jonevret made only two changes to the starting line-up that faced Highlands Park in a PSL match in Tembisa last weekend.
Patrick Phugwayo and Thabo Rakhale returned to the starting line-up, while Ntsikelelo Nyauza and Thembinkosi Lorch were both dropped.
Phunya Sele Sele caretaker coach Lehlohonolo Seema surprisingly left in-form Fiston Razak Abdul on the bench.
The Burundi international's brace helped the team beat National First Division (NFD) side Mbombela United 2-1 in the Nedbank Cup last 16 game a fortnight ago.
The Buccaneers slightly dominated the opening stages of the Cup encounter, but they struggled to penetrate the Phunya Sele Sele defence.
Wandisile Letlabika was looking solid in the heart of the Celtic defence, while keeping an eye on Bucs forward Tendai Ndoro, the current PSL top goalscorer.
Phunya Sele Sele were also struggling to create clear-cut chances. So, they resorted to long-range shots, hoping to beat Pirates goalkeeper Siyabonga Mpontshane.
Celtic midfielder Musa Nyatama missed the target from range in the 27th minute, before his teammate Atusaye Nyondo forced Mpontshane into an easy save with a long-range shot five minutes later.
The score was 0-0 between Pirates and Celtic during the half-time break following a quiet first-half which was tight defensively.
An entertaining second-half followed with the two sides creating good goalscoring opportunities as they looked to break the deadlock.
It was the home side, who took the lead through Abbubaker Mobara's well-taken goal much to the delight of the Bucs fans inside the Orlando Stadium.
The defender-come-midfielder fired home from a tight angle to make it 1-0 to Pirates with Thabo Matlaba grabbing an assist in the 54th minute.
Seema responded by introducing Kabelo Mahlasela for Deon Hotto, who was tightly marked by the Bucs defence most of the time.
Pirates second-half Thamsanqa Gabuza had a chance to score and double his side's lead in the closing stages of the match.
However, the lanky centre forward headed the ball straight into Celtic keeper Patrick Tignyemb's hands from close range.
Pirates were left to rue Gabuza's missed chance as Mahlasela punished the home side's defence, who were exposed with four minutes left.
Mahlasela restored parity with a low shot which beat Mpontshane to ensure that the game went into extra-time after a 1-1 stalemate in regulation time.
Gabuza made amends for his missed chance in the late stages of the second-half when headed home Rakhale's well-taken cross to make it 2-1 to Pirates in the 103rd minute.
The visitors had a chance to level matters again through Bucs academy product Mahlasela in the second-half of extra-time, but this time he missed the target and ultimately, Pirates won 2-1.