Read: Navy chief, defence ministry were at odds over accidents
Read: INS Sindhuratna mishap: Navy chief resigns as 2 go missing, 7 injured
A defence ministry spokesperson said he had stepped down taking moral responsibility for the recent accidents.
The mishap took place 50 km off the Mumbai coast early on Wednesday, while the boat was on a routine training exercise.
Seven sailors had suffered serious injuries and the two officers were declared "unaccounted for" after the accident.
A navy officer had said that the sailors had become unconscious after smoke started to fill the boat and were airlifted to a naval hospital in Mumbai.
Poll: Has this hurt the image of India's defence security?
Read: Twitter salute for Indian Navy chief for quitting with dignity
The submarine had undergone a refit at the naval dockyard in Mumbai barely two months ago, raising serious questions about the quality of upgrade it underwent.
The accident has taken the navy's mishap tally to 11 since INS Sindhurakshak, also a kilo-class boat, blew up and sank at a Mumbai harbour last August, killing all 18 men on board.
It was the third accident involving the navy's kilo-class (Type 877EKM) boats in less than six months.
In two years, navy may lose sub-sea edge over Pak
Navy may have itself to blame for series of recent accidentsMet PM before accepting Navy Chief's resignation: Antony