Peterson, who beat Khan in controversial circumstances to win the WBA super-lightweight and IBF light-welterweight belts in December, failed a drug test administered by the Las Vegas-based Voluntary Anti-Doping Association in March.
For the fight in Las Vegas to go ahead, Peterson needed to win a licensing appeal to the Nevada State Athletic Commission.
However, the ruling body was unable to hold a formal hearing until May 15, forcing the cancellation of the bout, Khan's promoters, Golden Boy Promotions, said.
"A failed pre-fight drug test administered by the Voluntary Anti-Doping Association coupled with the Nevada State Athletic Commission's legal inability to hold a formal hearing ... until Tuesday, May 15, has forced the cancellation of the event," Golden Boy Promotions said.
A disappointed Khan tweeted: "The fight is off! sorry everyone the only person to blame is @kingpete26 I'll be looking for a opponent to fight June 30."
Peterson tested positive for testosterone in an unscheduled doping test conducted by VADA in March.
"My understanding from my brief conversation with his (Peterson's) attorney was that when he was first told in April that he had tested positive for testosterone, he had no clue what it could be but that it had to be a mistake," Nevada Athletic Commission executive Keith Kizer told Reuters.
"Then the 'B' sample came back on May third positive as well and then at that time, I guess a light bulb went off that he had had some testosterone problems before the first Khan fight."
Peterson's team had promised to present its defence to the Nevada Athletic Commission earlier this week for the commissions chairman to then rule on whether or not the May 19 fight would go ahead.
"Unless he has got some really enlightening defence here, he will not be licensed for May 19," Kizer said.
Khan had been eagerly preparing for the rematch after losing his WBA and IBF belts in Lamont's home city of Washington on a split decision having been docked two points by the referee for pushing.
Khan's camp criticised the referee's actions and were shocked by television replays showing a mystery man in a hat talking with a judge ringside, prompting the WBA to order a rematch.
The 'mystery man' was identified as Mustafa Ameen, who is an IBF volunteer helping cash-strapped boxers. He denied interfering with the judges' scorecards.