The Falcons needed a late field goal to beat the Carolina Panthers and the Cardinals won in overtime against Miami while Houston marched to 4-0 for the first time in franchise history with a 38-14 win over Tennessee.
The San Francisco 49ers shut out a hapless New York Jets, winning 34-0 at the New Meadowlands while the New England Patriots avoided a third straight loss with a come-from-behind romp over Buffalo.
Atlanta rallied to win on Matt Bryant's 40-yard field goal in an outcome that may have left Panthers head coach Ron Rivera regretting the decision to punt on fourth and one with his team leading 28-27 and 59 seconds left on the clock.
Carolina quarterback Cam Newton, who threw for 215 yards and two touchdowns and also rushed for 86 yards and another score, fumbled on third down and although the Panthers recovered the ball, they fell a yard short of making the crucial first down.
Rather than see if Newton and the offense could make that yard, Rivera punted the ball back to Atlanta, with his special teams doing well to pin the Falcons back on their one-yard line.
Falcons quarterback Matt Ryan made the most of getting the ball back in his hands, throwing a superb 59-yard pass deep to receiver Roddy White who made a great catch setting up Matt Bryant for a game-winning 40 yard field goal.
Ryan completed 25 of 40 passes throwing for 369 yards and three touchdowns with one interception while White was on the receiving end of two of those touchdown passes as he put up 169 yards.
The Cardinals faced a more-dangerous-than-expected Miami, whose rookie quarterback Ryan Tannehill threw for 431 yards and one touchdown while wideout Brian Hartline put up a franchise-record 253 yards receiving.
A 15-yard touchdown pass from Kevin Kolb to Andre Roberts with just 22 seconds of regulation remaining saved the Cardinals and took the game into overtime.
After Arizona failed to make use of its first possession, Tannehill was intercepted by Kerry Rhodes at midfield and former Dolphin Jay Feely converted a 46-yard field goal for victory.
The Texans had a more straightforward victory to keep their unbeaten run intact as Danieal Manning and Kareem Jackson both returned interceptions for touchdowns off Titans back-up quarterback Matt Hasselbeck who came in early for the injured Jake Locker.
Arian Foster ran in a touchdown and quarterback Matt Schaub threw two touchdown passes as the Texans enjoyed the taste of victory against the franchise that was originally the Houston Oilers.
If Houston and Arizona are surprises at 4-0, it is equally unexpected to see New Orleans at 0-4.
The Saints' latest setback came with a 28-27 loss at Lambeau Field against a fired-up Green Bay Packers, determined to make up for Monday's infamous loss at Seattle.
An 11-yard pass from Aaron Rodgers to Jordy Nelson won the game for the Packers although the Saints had their chance.
Garrett Hartley put a 43-yard field goal attempt between the posts but a 10-yard penalty for holding by the Saints forced a re-kick and after a five-yard encroachment penalty against the Packers, Hartley missed a 48-yard kick.
Saints quarterback Drew Brees equalled Hall of Fame quarterback Johnny Unitas' record of throwing a touchdown pass in 47 consecutive regular season games.
There was more kicking drama in the Washington Redskins' dramatic 24-22 win in Tampa against the Bucs.
Redskins kicker Billy Cundiff missed field goal attempts from 41, 57 and 31 yards but he delivered when it mattered the most with his 41-yarder in the final seconds taking Washington to 2-2.
The 49ers utterly dominated the Jets in New Jersey on a tough day for home quarterback Mark Sanchez who managed just 103 passing yards, no touchdown and one interception.
San Francisco did most of their damage on the ground. They ground out 245 rushing yards and their four touchdowns were all run in including a 51-yard fumble return from Carlos Rogers.
The New England Patriots trailed 21-7 in the third quarter at the Buffalo Bills before a remarkable turnaround with six touchdowns as they stormed to a 52-28 win.
The Patriots, who had lost their previous two games, were helped by Bills quarterback Ryan Fitzpatrick throwing four interceptions while their own signal-caller, Tom Brady, threw for three touchdowns and 340 yards and also ran in a four-yard touchdown.
Minnesota continued their impressive early season form (3-1) with a 20-13 win at the Detroit Lions (1-3). The Vikings began in perfect style with a 105-yard touchdown return by Percy Harvin on the opening kickoff.
Quarterback Peyton Manning threw for three touchdowns and 338 yards in the Denver Broncos' 37-6 rout of the Oakland Raiders.