Unseeded Croat Cilic, who stands at 6’6” to Isner's 6’10”, produced when it counted in the tiebreaks on the blue clay at the Magic Box arena, winning the first 7-4 and the second 7-3 to set up a meeting with former US Open champion Juan Martin Del Potro who beat Mikhail Youzhny 6-4 7-6.
Neither player managed to break the other's serve as Cilic followed up his five-set victory over Isner at last year's Australian Open, their only previous meeting, with another narrow success.
Isner refused to blame the reverse on the blue clay, a controversial innovation at this year's edition of the Masters event that has prompted harsh criticism from players including world number one and defending champion Novak Djokovic and number two Rafa Nadal.
Djokovic slammed organisers and the ATP after his laboured second-round win on Tuesday, saying the Manolo Santana show court was too slippery and accusing officials of ignoring players' concerns.
"I lost because I didn't do what I should have done out there and he was better than me," Isner said. "I always felt that this tournament even with the red clay was the most slippery I have ever played on.
"It's tough for a big guy like me but my opponent was also a big guy and like I said he was just better and deserved to win and I did not deserve to win.
"I just didn't play particularly well today which has nothing to do with the courts."
In other second-round matches played on Wednesday, fifth seed David Ferrer beat the Czech Republic’s Radek Stepanek 7-6(4) 6-2, while 12th-seeded Frenchman Gael Monfils went through after Russia’s Igor Andreev retired inured after losing the first set.
There was more good news for the French as fourth seed Jo-Wilfried Tsonga, ninth seed Gilles Simon and number 14 seed Richard Gasquet all went through.
Tsonga beat American Ryan Harrison 6-2 7-6(4) while Simon toppled Guillermo Garcia-Lopez 6-4 6-3 and Gasquet beat Viktor Troicki of Serbia 7-5 6-3.
Alexandr Dolgopolov of Ukraine battled to victory over Italy’s Andreas Seppi 6-7(5) 7-6(5) 6-3, while Switzerland’s Stanislas Wawrinka beat Juergen Melzer of Austria 6-1 6-4.
Wawrinka's reward is third-round clash with defending champion and world number one Djokovic on Thursday.
Last year's beaten finalist Nadal , who is chasing a third straight clay title of the season, thrashed former world number three Nikolay Davydenko of Russia 6-2 6-2.
The Spaniard will face compatriot Fernando Verdasco after he beat Alejandro Falla of Colombia 6-7(4) 6-4 6-4.
Janko Tipsarevic beat Federico Delbonis 7-6(5) 6-7(6) 6-3 and Nicolas Almagro ousted Marcel Granollers 6-3 7-6(3).