which about 14 people were present at the time of the accident crashed into the water.
One of the victims was reported by some media to be a woman in her 30s, while the other two were male.
Rescue workers dived into the inky waters around the port -- one of the busiest in Italy -- in a frantic night-time search to find around 10 people believed to be missing.
Six others were reported seriously injured.
An employee of the Messina Line company based in Genoa which owns the Jolly Nero confirmed that "there was an accident when the ship was leaving the port. It ran into the tower, but we don't know why at this point, nor how many people are hurt."
The accident, which took place at around half past midnight (2230 GMT Tuesday), spooked Italians still reeling from the Costa Concordia night-time shipwreck off Giglio island in January 2012 which left 32 people dead.
Indictment hearings against six suspects in the cruise liner disaster began in Italy on April 15, with local authorities at the site of the shipwreck requesting compensation of more than 80 million euros ($105 million).
The main suspect is captain Francesco Schettino, who is accused of multiple manslaughter, causing a shipwreck, misinforming the coast guard after the crash and abandoning the ship during the rescue.
Emergency services at the scene in Genoa said it was not clear whether some of those still missing were trapped within the lift inside the tower.
Rescuers were using dogs trained to find people in earthquake zones to see if survivors were trapped under the rubble around the tower.
Around 3:30am (0130 GMT), the Jolly Nero was moved away from the crash site and rescue services were frantically searching the area. An AFP photographer saw a boat manned by firefighters.
Genoa mayor Marco Doria said Italians were in mourning after this "very serious port accident which struck an entire city".
The Italian container ship is almost 200 metres (655 feet) long, 30 metres (98 feet) wide, and has a gross tonnage of over 40,500. It was bound for Naples.
The ship's owner, Stefano Messina, who arrived at the port soon after the crash, choked back tears as he told journalists: "We are all utterly shocked. Nothing like this has ever happened before, we are desperate."
Prosecutors in the northwest Italian city opened an investigation while the Jolly Nero was sequestered by police, and the captain detained for questioning, reports said.
"I heard a terrible din and rushed out of my cabin," Roberto, the port's night watch, told La Repubblica newspaper.
"It was an incredible sight: the control tower was leaning perilously."
The impact happened during a shift change at the tower which meant that more people were present. The vast metal structure bent 45 degrees and a part of it fell into the sea.
"Based on the few details which have emerged so far, it was an incomprehensible manoeuvre which could only be explained by a mechanical failure," said Il Secolo XIX newspaper, based in Genoa.
The captain was quoted as saying: "Two engines seem to have failed and we lost control of the ship."