Les Bleus trailed 13-9 with more than half an hour to play after a slow first period during which the All Blacks crossed after just 93 seconds.
In the final 30 minutes winger Damian Penaud and replacement back Melvyn Jaminet crossed for tries, with full-back Thomas Ramos clinical with the boot as he amassed 17 points.
"It's incredible to score almost 30 against the All Blacks in an opening game," Dupont told reporters.
"We know we're capable of doing better than that, especially the first half.
"We told each other, we have to play better, they scored too easily.
"In a match that seemed average for us, we managed to score pretty much 30 points, which is a positive," the 26-year-old added.
Temperatures reached 29 degrees Celsius (84.3 degrees Fahrenheit) during the game at the Stade de France and a host of France players including No. 8 Gregory Alldritt seemed to suffer in the humid conditions.
"Obviously it was almost 30 degrees and in a stadium like the Stade de France, where there's not much fresh air, we felt that in the warm-up," Dupont said.
"It was humid, hard to accelerate, some guys had cramp.
"We felt it was like that for both teams. It took us a bit of time to adapt," 2021's World Rugby player of the year added.
Dupont's club and Test team-mate, hooker Julien Marchand, was replaced after 12 minutes with coach Fabien Galthie saying he had suffered a minor thigh issue.
Peato Mauvaka impressed off the bench with Pierre Bourgarit, who has won two Champions Cup titles in a row with La Rochelle, the third option in that position in the squad.
"We're hoping it won't be too bad for Julien," Dupont said.
"But you can see when Peato came on and did well, and Pierre in the stands, we have depth in that position," he added.
On a good note for Fabien Galthie's side, they only conceded four penalties compared to the All Blacks' 12.
"Our discipline allowed us to stay in the game, to stay in touch with New Zealand, and then get on top in the second half," the former France captain said.
"There are three or four boxes tick to be able to a win a game like this on, and discipline is really key."