Gary Cahill led England against Germany on Friday but the Chelsea defender is suspended for Sunday's match.
And Southgate confirmed at a press conference on Saturday that Manchester City goalkeeper Hart, currently on loan with Torino, would have the armband for the Wembley contest.
He said: "Joe is our most-capped international and showed maturity since last summer's (European Championship) disappointment."
Southgate also gave more detail about the injury which rules Manchester United defender Chris Smalling out of Sunday's contest.
"He picked up a knock towards the end of the game in Germany. After assessment, he wouldn't be ready for tomorrow," he said.
England's friendly defeat in Dortmund on Wednesday was marred by fans booing the German national anthem and singing distasteful songs.
Southgate said: "Supporters represent the country when they go away and the majority do that.
"Unfortunately some were chanting things the other night that were unacceptable - but we'd prefer to focus on those that support us well. It's a sell-out crowd tomorrow and we're grateful for that support."
On Lithuania, the England boss said: "We'll have to pay attention to their wide players. They're good in counter-attack and against Scotland scored with some good interplay with their front players.
Read the full article on eurosport.co.uk: Joe Hart named as captain for England qualifier