The Magpies had travelled to south Wales on the back of a run of four straight wins, but they fell to a comprehensive defeat as Nathan Dyer and Jonjo Shelvey found the net either side of a Mathieu Debuchy own goal.
Both incidents happened with the score at 1-0 and Pardew also felt Shelvey could have seen red after he appeared to aim a headbutt towards Debuchy during a heated exchange a few minutes before the ex-Liverpool man rounded off a superb individual display with a stunning, curling strike.
The Newcastle boss said: "Two big decisions went against us and that was crucial in the result.
"The first one was a penalty, no doubt in my mind about that. I felt that at the time and my players reaction told me everything I needed to know, and that should have told Howard as well.
"It hit his arm and it's going in the back of the net. That's a penalty isn't it?
"Again the second was for a foul on Gouffran. I thought Williams had no chance of winning that header, he has gone into the back of Gouffran and he just played on.
"I was disappointed in those decisions. I've told him that as well. He has given me his explanation and I accept that, and I don't agree with it.
"Then there is the incident with Jonjo, looking at video highlights I think that was disappointing for Jonjo as I think he is a terrific lad and that's unlike him, but again maybe you could argue he should not be on the pitch."
Swansea manager Michael Laudrup unsurprisingly had a different take on matters, particularly after his side recently conceded a controversial penalty to draw a game they should have won against Stoke.
Of the Shelvey incident he said: "Football is about temperament the most important thing keep on the right side, to not go over it and he didn't do that.
"It's important everybody sticks to the right side of the line."
Asked if he had headbutted Debuchy, Shelvey told BT Sport: ''That's what you think. Nowadays you can't do anything.
''I was trying to move to get out of the way. That's life at the end of the day.''
On the two penalty incidents, Laudrup added: "They claimed for two penalties. The one with the handball is what I would call 50-50. One of those that if you don't give it, it's okay.
"If I had been on the other side I would have claimed it, but if there are experts in penalties being given or not given it's Swansea.
"If he'd given it I would have accepted it.
"The other one I don't think was a penalty."
- Sports & Recreation
- Mathieu Debuchy
- Jonjo Shelvey
- Yoan Gouffran