The 31-year-old centre-back, who has signed a two-year contract with the south Yorkshire club, made just under 200 appearances for the Millers in a four-year spell between 2006 and 2010 before leaving to join the Shrews.
But despite the offer of a new deal and League One football with Graham Turner's side, Sharps has opted to rejoin the Millers as they move into their purpose-built New York Stadium this summer.
Rotherham boss Steve Evans said: "It has taken a long time to get Ian here. He has had a fantastic two years at Shrewsbury, he is someone we admire and respect and I am looking forward to seeing him in action."
Sharps becomes the third out-of-contract player from Town's promotion-winning squad to leave the club after Shane Cansdell-Sherriff and Nicky Wroe last week also snubbed new deals to join Preston.