A Reuters reporter saw a convoy of about 20 military transport vehicles and buses filled with armed rebels driving out of Kramatorsk where they had gone after apparently earlier fleeing Slaviansk 20 km (12 miles) to the north.
"Your order to free Slaviansk from the (separatist) fighters has been carried out," newly-appointed defence minister Valery Heletey was quoted by the presidential website as telling Ukrainian president Petro Poroshenko.
Armed pro-Russian separatists board a bus as they leave their positions in Kramatorsk in eastern Ukraine. (Reuters photo)
The rebellion in largely Russian-speaking eastern Ukraine has been a source of great tensions between the West and Russia. Moscow, which has already come under economic sanctions, denies Western accusations it has been backing the insurrection.
The Ukrainian flag had been run up on the main administrative building in Slaviansk, Heletey said, replacing the Russian one separatists had hoisted in early April when they seized key buildings in the city of 130,000.
Interior minister Arsen Avakov said a large number of separatists had fled in the face of sustained fire from Ukrainian forces.
"A significant number of militants have left Slaviansk ... along the way, our battle groups are greeting them. They are suffering losses and surrendering," he said in a statement on his Facebook page.
A source close to the rebels told Reuters they had been outnumbered by 50 to one. "(The Ukrainian forces) have greater numbers of troops and military hardware," the source said, speaking on condition of anonymity.
The rebels acknowledged the loss.
Aleksandr Borodai, a leader of the self-proclaimed Donetsk People's Republic, was quoted by the Interfax news agency as saying: "The punitive forces of Ukraine ... moved into a large-scale offensive.
"Given the disproportionate numerical superiority of the enemy troops, units of the armed forces of the Donetsk People's Republic were forced to leave their previous positions on the northern sector of the front."
Slaviansk has been the strongest redoubt of militants fighting government forces in eastern Ukraine.
Its re-capture represents Kiev's most notable military victory in three months of fighting in which more than 200 Ukrainian troops have been killed as well as hundreds of civilians and rebels.
Poroshenko's website said separatist fighters had come under mortar fire as they tried to break through government forces' lines. The separatists had lost one tank and other armoured vehicles, the statement said.
A Ukrainian paratrooper, Eduard, who was at an army checkpoint going into Slaviansk said: "It was a quiet night when suddenly a column of separatists appeared and began firing. They didn't have much luck. Artillery guns hit part of the town and part of those here."
"All those (separatists) who got to this checkpoint were killed. Those who were over there ran away," he said, gesturing higher up the road.
Some of the rebels appeared to have moved to Kramatorsk but they were pulling out of there quickly on Saturday apparently fearing air strikes from Ukrainian planes overhead.
People in the town said they had begun to pull out around 4am and about 100 of them had left the town.
A Ukrainian soldier walks near a destroyed armoured vehicle at Slaviansk in eastern Ukraine. (Reuters photo)
Uprisings in eastern Ukraine erupted in April as rebels took over state buildings, built a powerful arsenal of seized weapons and declared their independence from Kiev, calling the pro-European government illegitimate.
The crisis began when street protests ousted the Moscow-backed Viktor Yanukovich in late February for rejecting a landmark political and trade deal with the European Union in favour of closer ties with Russia.
Russia subsequently annexed Crimea and separatist revolts against the new Kiev authorities broke out with rebels declaring "people's republics" and saying they wanted to join Russia.
Talks in Berlin last week involving the foreign ministers of Ukraine, Russia, Germany and France had set Saturday as the day for talks between a "contact group" representing Ukraine, Russia and the OSCE security watchdog, and separatist leaders.
The talks were to have been aimed at working out an effective ceasefire observed by the opposing sides. Poroshenko declared a week-long unilateral ceasefire on June 20 which he renewed for a further three days.
But he refused last Monday to extend it any further, citing numerous violations by the rebels, and sent government forces onto an offensive against the rebels.
Russia has denied allegations by Kiev that it has been fanning the separatist rebellions by allowing weapons and fighters to cross over the long joint border to support the separatists. It has been pressing Poroshenko to engage in talks with the separatists and agree on a ceasefire.
Poroshenko said on Friday he had suggested a time and venue for the "contact group" to meet on Saturday. But given the sudden change in military dynamics in the region the meeting appeared now to be in question.
Hotbed of resistance
Slaviansk became a hotbed of resistance under the military command of Igor Strelkov, a Muscovite appointed as defence minister of the self-declared Donetsk People's Republic.
It has increasingly appeared to be going its own way independently of the rebel groups controlling Donetsk, the main industrial hub, and Luhansk, and many of the rebel violations of a government ceasefire which expired last Monday appeared to come from Slaviansk.
But on Friday Strelkov made an impassioned appeal for help to Russia, whom Kiev accuses of fanning the violence. He said without Moscow's aid the region the rebels lay claim to, known as Novorossiya (New Russia), would fall to Kiev's forces.
"Slaviansk will fall earlier than the rest," he wrote on a rebel website.
The government launched a new offensive against the separatists on Tuesday after Poroshenko allowed a 10-day unilateral ceasefire to expire without being extended.
Alla Belousova, who lives in Kramatorsk, said: "Pass on these words to (Russian President Vladimir) Putin: the people of the Donbass believed him for some reason when he said he would help. But now they (government forces) are killing peaceful civilians and if that is not genocide I would like to know what is.
"The (rebel) fighters are without shoes and don't have anything to fight with, she said.