The German's time of 1:32.330 was comfortably the quickest of a strangely uneventful session, a quarter of a second ahead of Red Bull's Sebastian Vettel.
Fernando Alonso was third for Ferrari after aborting his final lap of the session.
Lewis Hamilton qualified fourth quickest but will start ninth after incurring a five-place penalty for changing his gearbox.
It is the first time Mercedes have won back-to-back pole positions since Stirling Moss and Juan Manuel Fangio 1955 - Hamilton was on pole for the Chinese Grand Prix last week.
"Am I surprised? A little bit. It wasn't really clear who was the quickest car over one lap," said Rosberg of his second career pole. "I really want to kickstart my season. It has been a rough ride in the first three races. Today was my first real qualifying.
"It's difficult to say if we have enough pace to win the race tomorrow but for sure we'll try hard to win."
The early casualty from Q1 was Pastor Maldonado, the Williams driver squeezed out by his team-mate almost a year after winning the Spanish Grand Prix. He joined the usual Caterham and Marussia suspects.
Despite the early pace of Lotus in free practice, Romain Grosjean was the surprise casualty of Q2, Jenson Button celebrating making the top ten with an ironic cheer in his unfancied McLaren.
The top ten shootout was expected to be between the Red Bulls and Ferraris with but nobody bargained for a stunning late lap from Rosberg. And Ferrari will intriguingly begin the race with both drivers on different strategies, Alonso on the medium compound and Massa starting right behind his team-mate on the hard compound.
Arguably the stars of the show ere the Force Indias of Paul di Resta and Adrian Sutil, who start fifth and sixth with the revised grid which now reads Rosberg, Vettel, Alonso, Massa, Di Resta, Sutil, Webber, Raikkonen, Hamilton, Button.
- Motor Racing
- Sports & Recreation
- Lewis Hamilton
- Fernando Alonso