They went into the clash following consecutive losses to Leinster and Northampton and they were once again edged out in the south of France.
Dan Biggar slotted three penalties but was out-fired by Rory Kockett and Geoffrey Palis, who booted the French champions to a slender 15-9 victory.
The defeat leaves the Welsh region all but mathematically out of the Heineken Cup, while Castres maintained the chase on Pool One pace setters Leinster and Northampton, who clash on Saturday.
No side had ever qualified for the knockout stages following a pair of defeats from the first two rounds and the Ospreys were well aware they faced an uphill battle against the Top 14 champions.
They had just one full day of training with their Wales internationals and were without scrum rock Adam Jones but made a strong start nevertheless.
While they produced a desperately resilient performance, the Ospreys offered little in attack and paid the price for an extensive penalty account.
Wales fly-half Biggar was off-target with an early drop-goal but quickly made amends with a penalty.
Kockett should have responded moments later but was uncharacteristically off-target and missed once again after Aisea Natoga was sin-binned for a deliberate knock-on.
He eventually found his range and made it third time lucky on 15 minutes but missed once again as the Ospreys' indiscipline went unpunished.
Biggar fired the Ospreys back into the lead on 21 minutes but Kockett responded on the half-hour mark.
The French champions began the second half with far more purpose and they enjoyed a sustained period of pressure following a fine counter-attack from Brice Dulin.
The Ospreys survived that attack unscathed but Kockett soon nudged the hosts into the lead after their juggernaut pack roared into life at the set-piece.
They continued to pile the pressure on the Ospreys and a swift move looked to have released Paul Bonnefond in the left corner but the centre spilt the ball with the try line at his mercy.
Castres remained camped in Ospreys territory and launched wave after wave of attacks but the Welsh region held strong.
The pressure eventually told as Sam Lewis infringed at the breakdown within five minutes of coming on and paid the price for his side's incessant offences with a yellow card.
Kockett was successful with the ensuing penalty to hand his side a 12-6 lead after little more than an hour.
The Ospreys fought desperately in the closing stages and Biggar closed the gap on 73 minutes, to maintain his side's hopes of snatching a dramatic victory.
Palis instantly responded to restore the hosts' six-point advantage and they clung on for dear life. Ben John almost wriggled over in the closing stages and Palis was sin-binned for kicking the ball away.
The Ospreys forwards flexed their muscle as Castres continued to infringe, with Seremaia Bai also shown yellow, but the French side clung on.
- Sports & Recreation
- Dan Biggar