Kalmadi was presented a plaque by IAAF president Lamine Diack at the Congress in Pune.
The loss ended the veteran administrator's 13-year reign at the helm. Kalmadi, who was jailed and later released on bail for his alleged role in the Commonwealth Games scam, lost 18-20 in the presidential elections held on the second and final day of the AAA Congress at a five-star hotel.
All the 45 member countries of the AAA voted through secret ballot in the presence of international parent body IAAF chief Lamine Diack. There were seven invalid votes. Each country has one vote in the election.
Kalmadi, a former Indian Olympic Association President, will, however, continue to be a IAAF Council member till 2015 when the world body holds its elections.
Today's defeat also marked the end of 69-year-old Congress MP's career as a sports administrator as he has already quit from any such post in the country last year after he was charge-sheeted and jailed for his alleged role in the CWG scandal.
The electoral reverse has now led to speculation on whether he would now announce retirement from sports administration since he no longer has any posts.
He will though automatically become a life president of the AAA, an honorary post given to all past presidents, and can attend the Congress of the continental body without any voting rights.