Mubi, located in Nigeria's volatile northeast, where Islamist extremist group Boko Haram has carried out scores of previous attacks.
Last week in Mubi, Nigeria's military conducted a high-profile raid targeting the group, killing a senior Boko Haram figure and arresting 156 suspected members.
Motives for the gruesome off-campus attack however remained unclear, with some officials suggesting the massacre may have been linked to a recent student election.
Police have given an official death toll of 25, saying at least 22 of the victims were students. A school official said Wednesday that the death toll was at least 40, but he could not immediately say how many were students.
"Based on accounts from locals, at least 40 people were killed in the attack," the official from the polytechnic school said on condition of anonymity because he was not authorised to speak publicly.
"Twenty-five were the ones taken to the morgue. At least 15 of the victims whose families are in Mubi were taken away by relatives."
Abubakar Ahmed, head of the Red Cross in Adamawa state, where Mubi is located, said troops were going door-to-door looking for suspects on Wednesday.
Police spokesman Mohammed Ibrahim said security forces had blanketed Mubi, a commercial hub and university town located near the border with Cameroon.
"There's a heavy deployment of soldiers, police and (secret police) personnel in Mubi following the killings," he said. "They are combing all parts of the town in the hope of tracking down the killers."
According to Ibrahim, the attackers knew their victims and called them out by name in a student housing area off-campus of Federal Polytechnic Mubi, an ethnically mixed school with several thousand students.
Victims were shot or had their throats slit, he said.
Residents said it seemed the victims were both Muslim and Christian, but police had not commented as is often the case in Nigeria, where ethnic and religious divisions regularly lead to unrest.
The town had already been under a 3:00 pm to 6:00 am curfew in the wake of last week's raid, and it remained in place on Wednesday.
The suggestion that the killings were linked to the student election raised questions over how and why the dispute would have turned so violent.
There were suggestions of ethnic tensions between the mainly Muslim Hausas and predominately Christian Igbos involved in the vote, and a spokesman for the National Emergency Management Agency said some of the victims were candidates.
At the same time, Boko Haram has continually widened its targets and its attacks have become increasingly sophisticated.
Nigerian officials have been seeking to show success in the fight against Boko Haram with a number of raids and arrests. There had been a lull in major attacks in recent weeks.
The Islamist extremists have been blamed for more than 1,400 deaths since 2010 as part of their insurgency in northern and central Nigeria, Africa's most populous nation and largest oil producer roughly divided between a mainly Muslim north and predominately Christian south.
Maxwell Dukku, a spokesman for Adamawa's deputy state governor, said he would be surprised if the killings were linked to the student elections, while adding he had no further information on who was behind them.
"This kind of tragedy has never happened before," he told AFP. "Investigations are ongoing. There have been no arrests so far ... We really don't know why this has happened."
Boko Haram has claimed to be seeking an Islamic state in northern Nigeria, but its demands have repeatedly shifted and it is believed to include a number of factions with varying aims.
Imitators and criminal gangs are also believed to have carried out violence under the guise of the group.