The visitors fell behind in the 62nd minute when Chay Cooper curled a delightful strike into the top corner from the edge of the area after being picked out by Cameron McGeehan.
But Harrogate equalised less than three minutes later through Matt Daly, who finished clinically from close range from Sam Folarin’s pass.
And Folarin scored the winner in the 80th minute when he picked out the top corner with a fine effort from the edge of the area after Levi Sutton’s blocked attempt had fallen into his path.
Weaver said: “I’m really proud, especially with the reaction to going a goal down.
“We’ve been here (Colchester) the previous three years and lost games when we’ve played average, we’ve played well, we’ve played poorly and come away with nothing.
“We knew at half-time we could regroup and go again.
“The first half wasn’t a scintillating half, but it was a professional away performance where we conceded very few chances and we just needed to up it, on the ball.
“We’ve got a determination this year to be the best in the league to reacting to a disappointment.
“When we gave them the opportunity to go a goal up which they took advantage of, we saw a really positive reaction from the lads.
“They showed a real togetherness and turned it round and turned the game on its head.
“Sam (Folarin) made a telling contribution and Matty Daly’s finish spurred us on.”
Colchester United: Goodman, Greenidge, Mitchell, Hall, Egbo, McGeehan, Fevrier, Cooper (Edwards 75), Chilvers, Taylor, Ihionvien (Hopper 75). Unused substitutes: Dallison, Jay, Hornby, Kennedy.
Harrogate Town: Oxley, Ramsay, O'Connor, Burrell, Foulds, Cornelius, Sutton, Thomson, M Daly (Muldoon 88), J Daly (Folarin 58), March (Armstrong 74). Unused substitutes: Falkingham, Thomas, Sims, Sivi.
Referee: Paul Howard (London).