Arsenal had a perfect start when Olivier Giroud scored in the sixth minute, tapping in a pinpoint cross from Alex Oxlade-Chamberlain from close range.
But it turned into a miserable afternoon for Arsenal after that, as the Gunners ended up leaving the field to boos from its frustrated fans - having also seen two defenders get injured and another sent off to further deplete its already fragile backline.
"We start a new season with a defeat at home," Wenger said. "It's a massive blow."
Benteke started the comeback in the 22nd after Wojciech Szczesny brought down Gabriel Agbonlahor in the area. Szczesny saved Benteke's penalty, but the striker headed in his own rebound to equalize.
Villa were awarded a second penalty in the 61st when Laurent Koscielny was adjudged to have brought down Agbonlahor, although replays showed he made contact with the ball first. Benteke stepped up to the spot again and made no mistake this time.
Wenger criticized referee Anthony Taylor's performance, saying he didn't like the "spirit" in which the game was officiated, and the manager was especially perplexed about the second penalty decision.
"The linesman said to me he didn't give a penalty and he was at the level of the tackle," Wenger said. "If the referee didn't give the penalty at the start, why did he give it after? It was amazing."
With Arsenal getting increasingly frustrated, Koscielny was then given a second booking in the 67th for a foul on Andreas Weimann, reducing the hosts to 10 men. The Gunners still pressed for an equalizer, with Brad Guzan blocking a close-range shot from Rosicky and substitute Santi Cazorla's shot palmed onto the bar by the Villa goalkeeper.
But that exposed Arsenal at the back, and new signing Antonio Luna scored on his Villa debut with a quick counterattack in the 85th minute to seal the victory.
Villa manager Paul Lambert was pleased with the win, but stressed "we have miles and miles to go.
"This has given us a great start to the season," he said. "I think everyone thought we'd get no points from the first three games" against Arsenal, Chelsea and Liverpool.
Luna's goal capped a miserable day for Arsenal, which saw left back Kieran Gibbs substituted in the first half with a bloody gash to his forehead, and left back Bacary Sagna injured in added time when he landed on his neck after somersaulting through the air in a duel for the ball.
Emirates Stadium was more than half empty well before the final whistle as fans expressed their displeasure with both Wenger and Taylor.
"We want our fans to be happy. When you don't achieve that, you feel absolutely sorry and disappointment," Wenger said. "My job is to make the fans happy."
Wegner is under immense pressure to sign star players after chief executive Ivan Gazidis promised in June to "escalate" spending in the summer transfer market. But Arsenal so far has signed only one player, 20-year-old striker Yaya Sanogo, on a free transfer from second-tier French club Auxerre.
Arsenal tried to lure Gonzalo Higuain from Real Madrid, but the forward eventually signed for Napoli, while moves to buy striker Luis Suarez have been rebuffed by Liverpool.
Wenger defended Arsenal's transfer strategy, saying the club is willing to spend big, but only for the right players. He also stressed that "until we buy players, we have to win" with the current squad, which he said could have taken all three points Saturday.
He vowed to remain active in the transfer window until it closes on Sept. 2, saying "we were on the market before the game, and we're on the market after the game."
"If we don't spend the money, (it's because) we don't find the players," Wenger said. "I'm not the only one who works on that."
The club's disappointments in the transfer market have been compounded by a string of injuries to key players. Midfielder Mikel Arteta may be out up to six weeks with a thigh injury, while captain Thomas Vermaelen and Nacho Monreal (back), Abou Diaby (knee) and midfielder Ryo Miyaichi (rib) are all sidelined.
Oxlade-Chamberlain was replaced after halftime after appearing to injure his left leg in a challenge with Luna. Villa center back Nathan Baker also came off with an injury in the first half.
Wenger said Aaron Ramsey also picked up an ankle injury after being kicked by a Villa player, and that Rosicky had an unspecified injury.
The injury list is especially worrying since Arsenal heads into the first leg of a vital Champions League playoff round at Fenerbahce on Wednesday.