"We have so far retrieved 124 bodies (and) rescue campaign is still underway," Major Mahbub Hossain, a senior fire service official, told PTI over phone, adding that most of the bodies were found severely charred.
He said the fire service said the toll could rise.
Witnesses said it was the worst fire tragedy in recent years.
Fire service officials earlier said several workers of the factory were trapped inside and took shelter on the rooftop of the structure awaiting rescuers.
Efforts were still underway to extinguish the blaze with authorities mobilising several fire fighting units.
Television footage showed army troops and fire service rescuers bringing out bodies one after another from the debris as hundreds of people, including relatives of the victims, waited outside.