After a largely uneventful first half, Granit Xhaka opened the scoring with a deflected shot from distance, which looped over goalkeeper David de Gea.
And they doubled their lead three minutes later when Danny Welbeck headed home against his former club after a pinpoint cross from Alex Oxlade-Chamberlain.
Jose Mourinho introduced Marcus Rashford and Jesse Lingard to try to rescue the game, but they failed to bring the visitors to life.
The result means Arsenal remain in sixth place but are two points off United in fifth and six points off Manchester City in fourth, with a game in hand over both teams.
Analysis: 'Why Arsenal beat Man Utd but nobody really cares'
Listen: 'Ozil's performances have been lamentable'
Mourinho 8-1 Wenger
This victory not only reignites Arsenal's chase for a Champions League place, but also ends an unwanted record for Arsenal manager Arsene Wenger against Jose Mourinho.
The pair have endured a fractious relationship over the years, and in their previous 15 competitive meetings Wenger had never beaten a Mourinho team, with his only victory coming in the Community Shield in 2015.
Mourinho has won eight times, while the remaining seven were drawn.
Sunday's win ended the hoodoo and earned Arsenal only their 13th league win in 50 outings against the Red Devils.
There is still uncertainty about Wenger's future at the club, despite them still being in with a chance of Champions League football for the 21st consecutive season, and an FA Cup final against Chelsea coming up this month.
When asked by BBC commentator Jonathan Pearce about whether he would be interviewing Wenger next season, the Frenchman joked: "You want me to work for the BBC?"
Three minutes, two goals
Both teams had four shots on target during the match but it was Arsenal who made them count.
After a dull start to the second half, the Emirates came alive when Xhaka ran into space and took a shot from 30 yards out which was helped on its way by Herrera and out-smarted De Gea.
The Gunners kept up the pace and exactly three minutes and 11 seconds later, Welbeck had notched his second Premier League goal of the season.
Oxlade-Chamberlain's right-wing cross whistled over the box and Welbeck rose up between the United defence to land a bullet header.
Arsenal offered little in attack after the goals, but they did not need to.
All eyes on the Europa League?
Mourinho revealed before the match he intended to rest key players, with one eye on the Europa League semi-final second leg against Celta Vigo, in which his team have a 1-0 advantage. The European competition could represent United's best chance of reaching next season's Champions League.
The United manager made eight changes in total and his side never looked like causing a major threat.
Goalkeeper De Gea kept them in the match at half-time, producing a low stop to deny Aaron Ramsey's effort, before making a brilliant save from a rasping Oxlade-Chamberlain shot from distance.
United were undone by two quick goals early in the second half, and their only notable efforts after that came from Rooney and substitute Scott McTominay, who tried to poke home late on.
Man of the match - Aaron Ramsey
The end of the run for United
United's unbeaten league run has come to an end at 25 games (W13 D12), with losses on either side coming in London.
Alex Oxlade-Chamberlain has provided as many assists for Arsenal this season in the Premier League as in his previous five combined for the Gunners (seven).
This was the first time Oxlade-Chamberlain had assisted twice in the same Premier League game.
Danny Welbeck has scored in each of his last three appearances against Manchester United for Arsenal in all competitions.
Two English players combined for a Premier League goal for Arsenal for the first time since December 2014 (also Oxlade-Chamberlain for Welbeck, v West Ham).
Manchester United conceded more than once away from home in the Premier League for the first time since losing 4-0 to Chelsea in October.
Arsenal have won successive home Premier League games against the Red Devils for the first time since November 2001.
Arsenal manager Arsene Wenger on a possible top-four finish: "We want to win our games. Some teams that are safe continue to fight. Let's focus to win our games. Every win is important. We had a bad patch and seem to have recovered from it."
On Welbeck's goal: "That's the kind of goal you want from Danny. He has all the abilities a striker needs. Hopefully, that will give him a boost."
Manchester United boss Jose Mourinho: "We made eight changes. Of course, we knew we were not coming in our maximum power. That's a decision. We want to try to win the Europa League - it's more important than finishing fourth.
"The last trophy I won was three months ago. I didn't care about that. Thursday is the match of the season. I hope Old Trafford feels the same, because we need Old Trafford."
After the Europa League semi-final second leg on Thursday, Manchester United return to league action at second-placed Tottenham on Sunday, 14 May (16:30 BST).
Arsenal, meanwhile, have a midweek Premier League trip to Southampton on Wednesday (19:45 BST).