The Blues have been on a dismal run of just one win in eight league games which has seen them drop to tenth in the Premier League table. A 4-0 FA Cup defeat to Manchester City on Sunday also left the club out of both domestic cup competitions.
It comes amid a backdrop of ownership change, expensive and chaotic recruitment, and an injury crisis as the club overhauls its backroom staff, including the medical department.
Speaking ahead of a pivotal west London derby away at Fulham in the Premier League, Potter tried to explain just how challenging he has found his new job.
“Change is a challenge in any organisation. Listen, I am sure that the changes happened because of events outside of this and us. It’s not like there’s been a coup,” he told reporters, referring to sanctions on Roman Abramovich by the UK Government that forced a change of ownership at Stamford Bridge.
“This is what it is. We have to deal with the new and we have to build things up again because things have changed, things have gone and left. That’s part of the challenge to come and I understood that things would be difficult from a leadership perspective.
“It is a challenge, stimulating and ridiculously hard. I think it is probably the hardest job in football because of that leadership change and the expectation - because rightly where people see Chelsea.
“I obviously didn’t think we’d lose 10 first-team players but that’s where we are at. All I can do is speak to you guys honestly, give my perspective and understand the criticism if you lose.”
That honesty might not be enough to appease supporters who chanted for both Roman Abramovich and Thomas Tuchel in Chelsea’s most recent defeat.
There is frustration among supporters about the lack of impact from everything that is new aabout Chelsea. That is related to the owners, manager and new signings - who cost over £300million but have yet to make an impact.
Asked whether money has been wasted in the transfer market by Standard Sport, Potter explained that Chelsea are not just rebuilding the squad but the whole club.
“I think this club was run a certain way for 20 years and run really well,” he added. “I have a lot of respect for the previous owner and what they achieved and did. It is fantastic. Unfortunately, they’re not here anymore and you’ve lost all that leadership.
“It’s a new ownership group, everything has changed pretty quickly. When you look at everything that’s happened for the last six to 12 months, it is an incredible amount that’s happened. Sometimes that can manifest itself in different things and in different ways.
“I think it would be disrespectful, almost, to think that’s gone and expect it to pick up with all the new staff, structures and people. Lots of things went and you have to try to build it up again.
“At the same time you’ve still got Chelsea with the demands and expectations, but the reality of where the club is in terms of establishing itself as a well-run football club that functions well in a really competitive environment - maybe we’re not there yet.
“In my head, I think that’s quite easy to understand and get. I know there’s a lot of people that don’t see it that way. I am trying to explain but I also acknowledge that I am the head coach and when we lose that I’m to blame. They don’t really want to hear from me in terms of that perspective. They want to hear about a team and I get it.”
Despite Potter’s valid points about the problems at Chelsea since the takeover, the supporters are unlikely to tolerate many further poor results.
I think we are in a place where we can move forward.
The manager says he is doing what he can to turn results around amid a mutinous atmosphere, calling a meeting of the leadership group at Stamford Bridge to try to find the positivity and leadership that the team is lacking.
He noted: “I spoke really at length yesterday with Thiago [Silva], [Cesar] Azpilicueta, Jorginho, [Mateo] Kovacic - we had a really good conversation. They again showed their qualities as people.
“They were honest, articulated their concerns well. They articulated their positivity, the articulated their responsibility. And I think we are in a place where we can move forward.”