Gary Rowett's side returned to the top six with a hard-fought win at Reading on Saturday, and face home games against Swansea and Huddersfield, before six-pointers against West Brom (eighth) and Luton (fourth) after the international break.
"Our season is going to be defined in this last period," Cooper told Standard Sport. "Especially West Brom away, Luton home. I'm hoping we can get the results against the top teams, which we've been a lot better at this season.
“We have to make the most of our home games. Swansea are great in possession and they've caused us some issues in the past. And Huddersfield under [Neil] Warnock are fighting for their lives.
"We've got to bounce back from our defeat against Norwich at the Den and put some strong home performances on before the break."
Millwall have been in contention for the play-offs at this stage of each of the past three campaigns, only to fall away, but Cooper, who has played more than 250 games for the club, believes they are better equipped this season.
"We're just at that level now, within the top six or just outside," he said. "We know we've had a top side over the last few years, it's just about adding those players who can make that next step for us. The club has done well in recruitment and we look like we've got a strong enough squad."