It said the inspector general of police had been asked to carry out "an immediate and thorough investigation" into the incidents to identify the culprits.
According to police, the attackers broke off the heads of the statues of Gandhi and Britain's Robert Baden-Powell, the founder of the Scout movement.
Two other statues of local figures were also vandalised on Thursday in what appeared to be a coordinated crime, police said.
Batticaloa is an overwhelmingly Tamil majority town in eastern Sri Lanka. A Mahatma Gandhi statue also exists in Jaffna.