the MIC "has failed", reported NST Online, the website of the New Straits Times.
Vellu, 74, who has been MIC president since 1979, was re-elected for the ninth time some weeks ago. His term lasts till 2012.
The MIC, which has traditionally spoken for Malaysia's nearly two million ethnic Indian community, is a constituent of the ruling coalition Barisan Nasional (BN) since Malaysia became independent in 1957.
Mahathir warned that Vellu "will become a liability to the BN in the next general election, deue in 2013, because his leadership has failed".
The BN lost its two-thirds majority in parliament in the general election in March last year and control of five of 13 states, when a bulk of ethnic indians was perceived to have moved away from it.
The MIC was reduced to three seats in parliament and Vellu himself lost his 11th attempt at re-election to parliament.
Mahathir, who ruled Malaysia for two decades, when Vellu had served as a minister under him, said he was worried that the people's support for the Barisan would erode and the coalition would be the 'victim' because Samy Vellu was still leading the MIC.
"Samy Vellu must step down as party president because he must take responsibility for the party's failure in the last general election and for his own loss," Mahathir said in an interview with Bernama, the official news agency.
"If we look at other countries, whenever a leader fails, he resigns. In Japan he commits harakiri. We're not asking him (Samy Vellu) to commit harakiri. You (Samy Vellu) have failed to lead the MIC until you yourself lost," he said.
Mahathir had said in an interview on the eve of the MIC's general assembly last week that Vellu had "stayed too long".
Vellu flayed this remark after his team swept the MIC polls. An overzealous delegate demanded that a "shoe garland" be placed around Mahathir's portrait. Vellu suspended him and apologised to Mahathir.
"I will meet him (Mahathir) personally to explain the whole episode and extend my apology," Vellu said in a statement, The Star reported Friday.
Mahathir said he did not feel anything about the incident and added: "Nothing would have happened to me."
Mahathir, who stepped down in 2003, said he had "a right" to criticise Vellu, or anyone, for the sake of BN's future.
Known for his acerbic tongue, Mahathir had run a campaign against his hand-picked successor, Abdullah Ahmad Badawi.
Badawi stepped down as the prime minister in April after the coalition fared badly in the election.
"Malaysian leaders still do not understand that when they fail they should resign of their own volition, no need for others to tell them. When you fail, your party fails. You are responsible," he said.
Mahathir said: "Samy Vellu performed worse than Pak Lah (Badawi) but he did not want to relinquish the post. He should have resigned before things got to this stage.
"I've criticised Pak Lah, who's Samy Vellu that I cannot criticise?"
Badawi's successor and present Prime Minister Najib Tn Razak who addressed the MIC assembly last Saturday also asked the party leaders to "reach out to the Indian community" and win back their support.
It was not enough to be popular within party ranks, he told the MIC leaders in a apparent reference to internecine squabbles.
They should not be seen as "arrogant", Razak told them.