Arsenal were put to the sword in the first half by the clinical hosts, with goals from forwards Jean-Philippe Mateta and Jordan Ayew giving Palace a deserved 2-0 interval lead.
The Londoners improved after the break, with Emile Smith Rowe and Martin Odegaard missing big chances to get their team back in the match. Their wastefulness proved costly as Wilfried Zaha put the game to bed with a penalty 16 minutes from time.
Tottenham Hotspur's 5-0 home victory over Newcastle United on Sunday means they are fourth, ahead of their of their north London rivals on goal difference having played a game more.
Arsenal had been in fine form on the road in the Premier League prior to their trip to south London on Monday.
"Congratulations to Palace for the game they played but we made it impossible for ourselves with the way we competed," Arsenal coach Mikel Arteta told Sky Sports.
"I apologise to our supporters. We didn't have the presence today or the composure to dominate the situation so that is what I'm most annoyed with.
"First of all we need to put our hands up and get the criticism we deserve. Then we accept it, look ourselves in the mirror and look to the next game. This was not good enough."
Arsenal's 1-0 win at Aston Villa last time out was their fifth away league victory in a row - the first time they had been on such a run since May 2015.
But they struggled from the start to find any rhythm, with Palace very much the better side in the opening period.
It took the home side 16 minutes to break the deadlock, as Joachim Andersen's head sent a free-kick across the box for Mateta to steer in his fourth league goal of the season.
Eight minutes later and centre-back Anderson again turned provider, picking out a perfect pass for Ayew, who controlled well before finding the net for the first time since December.
Arsenal just could not create any openings of note in the first half, with Mateta going close to adding a third for Palace before the interval.
Content with a two-goal lead, the hosts sat back and invited pressure after the break. Smith Rowe should have done better from close range before Odegaard dragged his effort wide.
Zaha then broke on the counter and was awarded a penalty - only Raheem Sterling (23) and Jamie Vardy (21) have been given more penalties in Premier League history than the Ivory Coast forward - which he converted emphatically.
Arsenal substitute Eddie Nketiah did hit the crossbar but it would have been too little, too late.
A second defeat in their last three league games opens the door for Arsenal's top-four rivals and Arteta's side still have it all to do to secure a return to the Champions League for the first time since 2016-17.
Fifth-placed Arsenal are level on 54 points with Spurs and three points ahead of West Ham United and Manchester United, albeit with all their top-four rivals having played more games.
The scoreline at Selhurst Park did not flatter an impressive Palace, who continue to improve under former Arsenal captain Patrick Vieira, with the victory moving them up to ninth.
"I'm really proud," Vieira told Sky Sports. "We defended well and took our chances. We scored in important moments of the game. We needed a big team performance and we did it so we're really pleased.
"It is a good moment for the club but we know we need to be consistent in our performances and results."