The striker capitalised on a glaring error from Goran Popov as his hacked clearance fell straight to Hiram Boateng, who found Gayle and he simply wrong-footed Ben Foster with a left-footed strike from the edge of the penalty area.
It was recalled Gayle's fourth goal in only eight starts since his big-money summer switch from Peterborough and further strengthened his claims for a regular place in Tony Pulis' starting XI.
Pulis, who guided Stoke to the FA Cup final in 2011, is masterminding an impressive turnaround in fortunes since his appointment in charge of the Eagles.
This victory, his fourth in nine games, has now ensured the south London club are in Sunday's fourth-round draw for the first time in four years.
Substitute Marouane Chamakh converted a second goal with the last kick of the game deep into injury time, firing into an empty net.
Barry Bannan and Craig Dawson both hit the woodwork in an evenly-matched first half, before Palace goalkeeper Julian Speroni turned in a man-of-the- match performance after the break to deny dominant Albion at The Hawthorns.
Nevertheless, Baggies caretaker head coach Keith Downing fell to his first defeat in five games since taking the reins in the wake of Steve Clarke's sacking.
Downing made eight changes from the 1-0 victory over Newcastle on New Year's Day, with Foster, Gareth McAuley and Claudio Yacob the only survivors.
Frenchman Nicolas Anelka, subject of an ongoing Football Association investigation into his controversial 'quenelle' goal celebration, was not even included in the matchday 18.
Pulis, meanwhile, included five new faces from their 1-1 draw against Norwich - Gayle, Bannan, Boateng, Jonathan Parr and Jonny Williams all getting the nod.
The impressive Saido Berahino, who bagged Albion's late winner from the penalty spot against the Magpies to secure a first Barclays Premier League win in 10 attempts, comfortably volleyed over the first effort of the game after 10 minutes.
The visitors quickly came close at the other end as Bannan rounded Foster only to crash his shot against the near post.
A free-flowing encounter then saw Matej Vydra latch on to Zoltan Gera's pass before firing straight at Speroni.
And it was no surprise when the deadlock was broken after just 22 minutes.
Popov's wayward clearance went straight to Boateng, who found Gayle and he wrong-footed Foster into the bottom corner from the edge of the penalty area.
Albion almost conjured up an immediate reply but Dawson was unlucky to see his header loop off the top of the bar.
Openings dried up as the first half wore on, with the Baggies toothless despite neat approach play and Palace doing enough - Vydra's half volley testing Speroni just before the break.
The Palace stopper collected two drilled efforts from Berahino at the start of the second half before Yannick Bolasie wasted a great chance to double the away side's lead, somehow heading over unmarked from four yards.
Speroni produced the save of the match after an hour to tip Vydra's low shot around a post having seen it late.
And the Argentinian was at it again soon after to keep out another effort from Berahino.
West Brom have a rich history in the world's oldest domestic cup competition having lifted the trophy five times but they could find no way through against a resilient Palace defence.
And, despite having beaten the Eagles 2-0 in the Premier League earlier this season, they were left to reflect on a third third-round exit in four seasons.
Indeed, after an offside Gayle had poked home Bolasie's goalbound shot late on, Chamakh did add a second from Bolasie's pass deep into injury time.
- Sports & Recreation
- Hiram Boateng
- Julian Speroni
- Tony Pulis
- Goran Popov