The Scots came into the game off the back of two defeats, but started the game very well, scoring three goals in the space of four minutes to shock the visitors.
Patrik Twardzik opened the scoring after 11 minutes, with Callum McGregor and John Herron also on target.
The Dutch side clawed a goal back just before half-time through Geraldo Antonio but a second-half fight back wasn't enough as Celtic held out for a 3-1 victory.
PSV were reduced to 10 men in the dying seconds when Mohamed Rayhi saw red following a scuffle.
The win takes the Scots above Aston Villa into third place in Group Four.