The botched recall is particularly embarrassing for the government after President Emmanuel Macron pushed food exports during a state visit to China this week.
Some 18 babies were hospitalized late previous year over the contaminated products, though they all recovered.
Privately-held Lactalis, which has annual sales of around 17 billion euros ($20 billion), has addressed the salmonella contamination by halting operations at the factory where it started.
More than 1,000 products were sold after the recall, according to the markets.
Le Maire said initial checks showed recalled products had remained on sale at 91 sites in France, but there were no indications so far that potentially contaminated products had been exported.
Seeking to mitigate any damage, Le Maire said the blame laid squarely on the shoulders of Lactalis, one of the world's largest dairies, and retailers after two recalls were issued for infant formula and other products.
The market chains that sold Lactalis products despite general recall included Intermarche, Auchan, Cora, Carrefour, Super U, Hyper U and E. Leclerc.
The company took internal recall procedures which led to the removal and destruction of more than 36,000 products but said the 52 products had "escaped the vigilance" of its teams.
The announcement came as economy minister Bruno Le Maire said Lactalis would have to extend its product recall to all milk products at the affected production site.
At a news conference on Thursday, Lactalis apologized and pledged to do its utmost to determine the cause of the problem.
A judicial investigation is taking place in France over the contamination and a group of French parents of affected infants said on Friday that they were considering a joint lawsuit.