There was no immediate claim of responsibility for the attack, which is being investigated by anti-terror police.
Greek Prime Minister Antonis Samaras and Public Order Minister Nikos Dendias spoke to German ambassador Wolfgang Dold by telephone after the incident.
Attacks against diplomatic targets, banks and foreign companies are regular occurrences in Greece. They are often blamed on far-left or anarchist groups and in most cases only cause material damage.
Anti-German sentiment has been on the rise in Greece in recent years over EU paymaster Germany's perceived role in pushing for severe austerity measures in return for the country's 240-billion euro ($330 billion) bailout.
The German ambassador's residence in the Athens suburb of Halandri was also the target of a rocket attack in 1999, in an incident claimed by the extremist left-wing November 17 group, which has since been dismantled.