The Russian managed to nullify the big serving Croat – despite Karlovic firing 15 aces – to secure a 7-6 (4) 6-3 victory.
Davydenko has one of the best records on the Tour against Nadal having won their last four meetings and holding a 6-4 record overall.
However, Nadal has won the last three meetings on clay, although this match will be played on Madrid's controversial new blue clay.
"He has less confidence against me on hard court; I have beaten him more times there," Davydenko told the ATP website.
"On clay I have never beaten him. He is the best player on clay for sure. That's where he starts to fight and play different tennis. He runs different and it's very tough to beat him on clay.”
In the only other main draw match Sunday, South African Kevin Anderson beat Alex Bogomolov Jr. 6-3 6-2 to set up a second round clash with Tomas Berdych.