During Jo Konta’s rubber with Sorana Cirstea in Constanta, which she eventually won 6-2 6-3, Nastase allegedly used aggressive language towards the match umpire and both the British captain Anne Keothavong and the No 1 player.
He was escorted from the court and play was suspended for several minutes.
Simona Halep, the home No 1 player, who had earlier beaten Heather Watson 6-4 6-1, had to address the crowd after the atmosphere turned sour with Konta in tears.
Former world number one Nastase had earlier made highly offensive comments about Serena Williams and her pregnancy and became involved in a row with a PA reporter.
Nastase, who was later kicked out of the tie by the ITF for "unsportsmanlike conduct" on court, told Romanian television news channel, Digi24, that his remark was not racist and that he has a very good relationship with Williams.
"This was a joke," he said. "If someone didn't understand it, they lack humour... they have their own humour and we have our humour."
ITF president David Haggerty said an investigation would be launched.
"This is unacceptable behaviour by a Fed Cup captain. No player, official, member of the media or fan should have to endure any kind of abuse, and Mr Nastase will rightly play no further part in this tie," he said.
"A formal investigation is already underway and any decision or sanction will be made by the ITF's Adjudication Panel. We are unable to comment further on an ongoing investigation."
Nastase was on court in Romania and unavailable for immediate comment.
George Cosac, President of the Romanian Tennis Federation, said the RTF had yet to receive guidance from the ITF. "As soon as we are given the conclusions, we will take a decision."
"The ITF does not tolerate discriminatory and offensive language and behaviour of any kind," the International Tennis Federation said in an emailed statement.
Additional reporting by Reuters.
"We are aware of alleged comments made by Romanian Captain Ilie Nastase and have begun an immediate investigation so that we have the full facts of the situation before taking further and appropriate action."
Read the original article on Eurosport: Konta reduced to tears after Nastase verbally abuses British team and umpire