Volkswagen chief executive Martin Winterkorn.
The probe will look at "allegations of fraud in the sale of cars with manipulated emissions data", the prosecutor's office said.
Mr Winterkorn quit last week after almost nine years at the helm of VW, saying he had no knowledge of the manipulation of emissions results.
Regulators in the US had found "cheat" software in some diesel engines.
In the German legal system, anyone can file a criminal complaint with prosecutors, who are then obliged to examine them and decide whether there is enough evidence to open a formal investigation.
In this case, following the US revelations about the rigged tests, prosecutors in Braunschweig, near VW's headquarters in Wolfsburg, received about a dozen complaints, including one from Volkswagen itself, said spokeswoman Julia Meyer.
Over the weekend, German media reported that some of Volkswagen's own staff and one of its suppliers had warned years ago about the illegal use of so-called "defeat devices" to detect when a car was being tested and alter the running of its engines.
Audi admissionThe head of VW's Porsche division, Matthias Mueller, was appointed on Friday as Mr Winterkorn's successor.
VW has apologised for cheating emissions tests, but says that some 11 million cars across the group may can contain the computer code.
On Monday, VW-owned Audi said 2.1 million of its cars worldwide were fitted with the software.
Some 1.42 million Audi vehicles with so-called EU5 engines are affected in western Europe, with 577,000 in Germany, and almost 13,000 in the US.
Affected models include the A1, A3, A4, A5, A6, TT, Q3 and Q5, a spokesman told the Reuters news agency.