Djokovic responded angrily to being handed a time violation, yelling “you have to understand the game” at the umpire.
The outburst overshadowed an entertaining match, in which Chardy ran Djokovic close before eventually succumbing 6-4 6-3 in 1 hour and 24 minutes.
A two-time winner at Crandon Park, Federer did not visit Miami last year, taking an extended layoff before starting his season. But he looked quite at home, needing just 74 minutes to slay the 6-foot, 11-inch (2.11m) Croatian in a slugfest that featured just one break of serve.
"I was able to play a very clean match on my service games throughout, make sure I had a lot of first serves in, make sure I stayed aggressive from the baseline," said Federer. "I think it worked really well, hardly any unforced errors.
"There are no long rallies. It's just like more penalty shootout like in soccer.
"It's about quick reaction, not getting frustrated and feeling that the returner is the one who has got less pressure."
Federer, who had been dismissed as a fading force following last year's disappointing campaign, has hit back at his critics on the court, clinching the 78th title of his career in Dubai while reaching the finals in Brisbane and last week in Indian Wells, where he lost to Novak Djokovic.
It was another dominating performance by the re-energised Swiss maestro, particularly in the opening set where he committed a single unforced error.
Federer broke to open the match on a Karlovic double fault and did not surrendered a point on his serve until serving out the set at 40-0.
The 32-year-old Swiss continued his near perfect play in the second set finishing with a total of three unforced errors but could not break the big-serving Croatian, who blasted 14 aces past the 17-time grand slam winner.
But Karlovic's big serve was not able to save him in the tie-break as Federer coolly clinch the match 7-4.
Fourth seeded Spaniard David Ferrer and ninth seeded Frenchman Richard Gasquet both had easy passage to the next round.
Ferrer, who lost last year's final to Andy Murray, returned to action after sitting out the Masters series event in Indian Wells with a left abductor strain and showed no hint of rust as he cruised past Russian Teymuraz Gabashvili 6-4 6-0.
Gasquet made equally quick work of Alejandro Gonzalez dispatching the 83rd ranked Colombian 7-6 (7) 6-4.
- Sports & Recreation
- Roger Federer
- Ivo Karlovic