Investigations are continuing into the ugly scenes that scarred Saturday's FA Cup semi-final defeat to Wigan at Wembley.
While the vast majority upheld a general improvement in behaviour from the Lions fans since the ugly days of the 1980s, 12 were arrested for their part in the disgraceful outbreaks of fighting.
Millwall have vowed to take the strongest action against those found to be responsible.
But Jackett feels there is only so much the club can do.
"We will continue to work hard," he said.
"Whether we ever do it (get rid of the reputation) is a tough call.
"I have been here a number of years and am very proud of the way everyone has worked at the club.
"There has been a considerable amount done. The board of directors have worked in every direction they possibly can, making sure the players go out into their local community.
"It was a disappointing day."
- Sports & Recreation