The home side's new French signing Alexandre Lacazette had given them an early lead.
Lacazette is the seventh Arsenal player to score on his first Premier League appearance. "He is an intelligent player and we have a game backed on mobility and technique and I think he can integrate with that well", said Wenger, who is now entering his 22nd season at the Gunners' helm.
"We were punished on every single mistake by a Leicester team that was very efficient going forward".
Ramsey fired home the leveller in the 83rd minute before Giroud brought the home supporters to their feet two minutes later when he scored the victor. The manager's second-half tactical tweak left plenty confused as he shifted to a 4-2-3-1 system with a right-back playing on the left, two left-backs at centre-half and a midfielder at right-back - but it worked as his two substitutes scored the vital goals that turned the game on its head.
Only 105 seconds it took the former leader of the Lyon's Alexander Lacazette to score his debut goal in the Premier League.
That lead, however, lasted just three-minutes as Shinji Okazaki headed home Leicester City's equaliser after Harry Maguire brilliantly headed a deep cross into the area back across goal from the far post for his striker to finish past Petr Cech.
For the first time in the history of the English championship kicked off on a weekday.
Danny Welbeck leveled proceedings deep in added time, scrambling the ball home from close range after it had been squared to him by Sead Kolasinac.
Jamie Vardy scored a brace for Leicester City in the defeat.
Substitute Aaron Ramsey rifled home from close range to level before Giroud - introduced alongside the Wales global - headed in an 85th-minute victor to settle a breathless encounter.
Leicester showed some of the form that won them the title in 2016 and helped stave off relegation last term but in the end they were undone by canny France global Giroud who has been linked with a move away from Arsenal all summer.