The US drone strike that accidentally killed two hostages in Pakistan exposes intelligence shortfalls that former and current US officials say appear to be growing more frequent as militants expand their safe havens and as Washington gathers less on-the-ground human intelligence.
Obtaining timely intelligence on hostages has always been difficult, especially in volatile regions where the United States has limited access and where militants have well-established operations.
But as unrest spreads, militants are acquiring more safe havens, from Pakistan and Afghanistan to Syria, Libya, Yemen and Iraq, complicating and often hampering US intelligence-gathering. This is especially so in the wake of the Arab Spring as militants exploit the vacuum left by shattered institutions.
That has forced American intelligence operatives to become more dependent on electronic eavesdropping and spy satellites rather than using informants and on-the-ground human intelligence, say the former and current US officials.
The inadvertent killing of American doctor Warren Weinstein and Italian aid worker Giovanni Lo Porto in a January US drone strike, acknowledged by US President Barack Obama on Thursday, follows two failed US attempts in the past nine months to rescue Western hostages. Those efforts apparently relied on dated or incomplete information.
Last July, US Delta Force commandos swooped into eastern Syria to try to rescue US journalist James Foley and other hostages, only to find they had been moved. Foley was later executed by his Islamic State captors.
A December attempt to free American photojournalist Luke Somers and South African teacher Pierre Korkie in Yemen failed when their al Qaeda captors were alerted to US commandos' approach and executed them.
Of all those regions, few have remained off limits for as long as Pakistan's rugged northwest North Waziristan, where Weinstein and Lo Porto were held and where a generation of Taliban and al Qaeda militants have built a stronghold for launching attacks on US-led forces in Afghanistan.
Some former US officials say the problem is too few US informants on the ground in danger zones such as Pakistan or Yemen. "You can't do intelligence operations without HUMINT," said one former senior US intelligence official, using the acronym for "human intelligence."
Rescue missions in enemy territory are inherently risky and, officials say, based on imperfect information.
"The rule is, you almost never know where these guys are," said a US official, speaking on condition of anonymity.
"No one silver bullet"
The latest killings re-ignited criticism from hostages' family members about White House efforts to protect their loved ones, and stoked controversy over the lethal drone programme.
In the drone strike that killed Weinstein and Lo Porto, sources said the Central Intelligence Agency had no idea the two were being held there despite hundreds of hours of surveillance of the al Qaeda compound.
White House spokesperson Josh Earnest said that before the strike, US government assessments had arrived at "near certainty" that civilians would not be harmed. An internal review of the operation is underway to see if reforms are needed to prevent similar incidents, Earnest said.
Whether mistakes were made or not, it is very difficult for US spy agencies to acquire timely information about where and how hostages are being held, the officials said.
"It's a very complex proposition," requiring the stitching together of multiple streams of intelligence from various data collection methods, said Dane Egli, a former senior White House advisor for hostage policy under President George W Bush. "There's no one silver bullet."
To militant groups, hostages are an extremely valuable commodity and kidnappers make their captives' security a top priority, the officials said.
Egli said that opportunities to learn information from local inhabitants or interrogating detainees have been reduced as the United States has withdrawn troops and intelligence assets from Iraq and Afghanistan. Another obstacle is the expansion of safe havens and ungoverned spaces, from the Afghanistan-Pakistan border to Yemen.
"Any time they have secured real estate ... it's harder for us to penetrate the (US military) Special Forces for us to do a surprise mission" and attempt rescue, Egli said.
Sometimes there is virtually no information at all. American journalist Austin Tice disappeared in Damascus in August 2012, and has not been heard from other than a brief video that surfaced five weeks later.
US officials have given Tice's family no indication they know where he is, a person familiar with the situation said on Thursday.