Adebayo Akinfenwa opened the scoring after City captain Dan Seaborne saw red for a foul on former Sky Blues forward Cody McDonald.
Callum Wilson equalised for Coventry from the spot shortly after half-time before Myles Weston restored the hosts' lead after 56 minutes with his first goal of the season.
Akinfenwa then notched his second penalty after 75 minutes after he was fouled by Andrew Webster - one of the decisions which angered Pressley.
But Gillingham's nerves were tested when Coventry captain Carl Baker netted a fourth spot-kick of the evening with seven minutes left before Jake Hessenthaler eventually sealed a thrilling victory for Peter Taylor's side late on.
However, Pressley sympathised with defender Webster and said: "I'm going to see the referee, and I've watched the DVD to ensure I was correct in what I was thinking.
"If you look at the second penalty that was given to them, the referee let play go on, the player missed the chance, then signalled for the penalty.
"I was at a meeting about this, and you can't have it both ways - he should make one decision or another.
"And for the Callum Wilson penalty, the rule is not last man, it's denying a goalscoring opportunity.
"He was ahead of the last two defenders, he was brought down [and Leon Legge should have been sent off]. The decisions are beyond me.
"I think that the team's performance was brilliant, though. Throughout the 90 minutes, even with 10 men, we were the dominant team, our desire to do things was terrific and at one-one there was only one team who was going to win the game."
- Sports & Recreation
- Adebayo Akinfenwa
- Andrew Webster
- Sky Blues