The Islamist militia, which has been fighting against US-led Nato troops and the Afghan government for 12 years, broke off initial contacts with the Americans last year and have long refused to negotiate with Kabul.
Their announcement came just hours after Afghan government forces formally took over responsibility for national security from a Nato combat mission scheduled to leave the country next year.
An AFP photographer said Taliban representatives and Qatari officials opened the "political bureau of the Islamic Emirate of Afghanistan" in Doha.
Taliban spokesman Zabiullah Mujahid told AFP that the office was intended "to open dialogue between the Taliban and the world".
In a statement, the Taliban said the office would help to build relations with the world, allow them to meet other Afghans and to contact the United Nations, other agencies and the media.
"We support a political and peaceful solution that ends Afghanistan's occupation, and guarantees the Islamic system and nationwide security," it said, making no direct reference to peace talks.
The United States immediately welcomed the decision and senior officials said they hoped to meet their Afghan foes within days.
"I think the United States will have its first meeting with the Taliban for several years in a couple of days in Doha," a senior US official told reporters.
"I would expect that to be followed up within days by a meeting between the Taliban and the High Peace Council, which is the structure that President Karzai has set up for talks of this nature," he added, dubbing the move the "beginning of a very difficult road".
The United States has led growing pressure for a political solution to end the violence in Afghanistan, and the new Taliban office has been touted as a tool to help facilitate talks between the militants and the government.
Afghan President Hamid Karzai, who has long called for peace talks, announced earlier Tuesday that he had ordered government envoys to travel to Qatar to try to open negotiations.
It was not immediately clear whether the Taliban would meet his representatives.
"We hope that with the opening (of the office)... the peace talks between the HPC (High Peace Council) and the Taliban start as soon as possible," he told the security handover ceremony.
The security handover at a military academy outside Kabul marked a major milestone in the long and bloody US-led combat mission that began after the 9/11 attacks in 2001.
Karzai pledged that Afghan forces were ready to take on the insurgents, but persistent violence was highlighted when a bomb targeting a lawmaker killed three people in the capital just before the ceremony began.
"From here, all security responsibility and all security leadership will be taken by our brave forces," Karzai said at the event, the timing and location of which had been kept secret due to fears of attack.
Nato chief Anders Fogh Rasmussen, who also attended the ceremony, said that by taking the lead in security, Afghan forces were completing a five-stage transition process that began in March 2011.
"They are doing so with remarkable resolve," he said. "Ten years ago, there were no Afghan national security forces... now you have 350,000 Afghan troops and police, a formidable force," he said.
The handover of the last 95 districts from Nato to Afghan control includes areas in the south and east where the Taliban have concentrated their insurgency since 2001.
Doubts remain over the ability of Afghan forces, and the 98,000 foreign troops will retain an important function in training, logistics, air support and in combat emergencies.
Concern over capacity have been fuelled by high rates of desertion and fears for the future of foreign aid post-2014.
"The reality is Afghan forces are not dreadful, but they're probably not sufficiently capable to drive the war to a conclusion," said Stephen Biddle, professor of international affairs at George Washington University.
"My guess is they will be able to maintain the stalemate, provided the US pays their bills," he told AFP.
The Taliban have a proven ability to strike at Kabul as the country prepares for presidential elections and the Nato withdrawal next year.
Haji Mohammad Mohaqiq, a leader of the ethnic Hazara minority, was unhurt in Tuesday's bomb attack but three civilians were killed and 24 others, including some guards, were wounded.
On Friday, the US commander of the Nato mission in Afghanistan warned that gains secured over the last 12 years would be lost if donor nations cut back support after the foreign withdrawal.
According to independent website icasualties.org, at least 3,336 foreign troops have died since the start of operations in 2001.