Japan were out to build on a huge comeback victory over Germany to kickstart their tournament but left matchwinner Takuma Asano on the bench as Hijame Moriyasu made five changes.
Costa Rica made two changes and switched to a five-man defence after the hammering at the hands of Spain.
Ritsu Doan, who netted the equaliser against Germany, was one of those introduced by Japan and his fine cross into the six-yard box teased an early opener, but with the goal gaping it went unmet.
While the Blue Samurai lacked a cutting edge, it was ambition largely found wanting for La Sele as they remained content to knock the ball around at the back.
A rare move forward after the half-hour mark drew a half-clearance to Joel Campbell, who lofted a hopeful poke at goal over the bar.
The end of one of the dullest halves in Qatar so far prompted Moriyasu to make two half-time changes, switching to a 3-4-3 formation with the introduction of Asano.
A momentary surge faded as Japan looked less like the swashbuckling comeback kings that downed Germany and more the raw, wayward young team that many expected to show up at this World Cup.
As the clock ticked, they became guilty of the lack of end product that did for the Germans and Costa Rica snatched all three points in the final ten minutes.
Keysher Fuller stepped up to produce the winner after scrappy play from Japan surrendered possession by their own box and a snapshot from the edge of the area managed to catch Shuichi Gonda napping and nestle in the top corner.
Costa Rica saw out the win with men behind the ball, their first shot on target in 170 minutes at this tournament earning their first World Cup victory in a 90-minute match since beating Italy in the 2014 group stage.