O'Connor, who currently works under director of rugby Richard Cockerill at Aviva Premiership club Leicester, has agreed a three-year deal and will link up with Leinster on July 1.
He will succeed Joe Schmidt, who was recently appointed as the new Ireland coach in succession to Declan Kidney.
O'Connor, 42, is a former Australian international who coached Australia A before joining the Leicester staff in 2009. During his time at Welford Road, Leicester have won two Premiership titles and reached the 2010 Heineken Cup final. Tigers are also through to this season's title play-offs and will play Harlequins next Saturday.
O'Connor said: "I am delighted to join Leinster. The playing staff, coaching team and facilities are first-class, and I have been hugely encouraged by the hunger within the province to push on despite enjoying an unprecedented period of success on the field in recent years.
"I look forward to working with the province's established and burgeoning talent from the renowned Leinster academy, and I am also pleased to work with the current coaching team who have played such a key part in the province's success over the last four years.
"At this time my focus remains with Leicester, but I would like to thank the Leinster board for this vote of confidence. My family and I are hugely excited about starting this new chapter in our lives."
And Leinster chief executive Mick Dawson added: "We are delighted that Matt has agreed to become head coach, and we believe that he will be carry on the fine work which has been done over the course of the last eight years.
"Matt joins Leinster with an impressive pedigree. We were impressed by his vision for Leinster Rugby, and speaking to him it is clear that he is excited about the opportunity. We look forward to working with him over the coming years.
"Obviously, we are still in contention for three trophies - the Amlin Challenge Cup, RaboDirect PRO12 and British & Irish Cup, and all of our focus will be on trying to end the season on a high in all three competitions."
- Sports & Recreation
- Leinster Rugby
- Richard Cockerill