The US government is quietly funding research to prevent eavesdroppers from seeing who the US is spying on, even as the Obama administration considers ending the storage of millions of phone records by the National Security Agency, The Associated Press has learned.
The Office of the Director of National Intelligence has paid at least five research teams across the country to develop a system for high-volume, encrypted searches of electronic records kept outside the government's possession. The project is among several ideas that could allow the government to store Americans' phone records with phone companies or a third-party organization, but still search them as needed.
Under the research, US data mining would be shielded by secret coding that could conceal identifying details from outsiders and even the owners of the targeted databases, according to documents obtained by The Associated Press and interviews with researchers, corporate executives and government officials.
The administration has provided only vague descriptions about changes it is considering to the NSA's daily collection and storage of Americans' phone records, which are presently kept in NSA databanks. To resolve legal, privacy and civil liberties concerns, President Barack Obama this month ordered the attorney general and senior intelligence officials to recommend changes by March 28 that would allow the US to identify suspected terrorists' phone calls without the government holding the phone records itself.
One federal review panel urged Obama to order phone companies or an unspecified third party to store the records; another panel said collecting the phone records was illegal and ineffective and urged Obama to abandon the program entirely.
Internal documents describing the Security and Privacy Assurance Research project do not cite the NSA or its phone surveillance program. But if the project were to prove successful, its encrypted search technology could enable the NSA to conduct secure searches while shifting storage of phone records from agency data banks to either phone companies or a third-party organization.
A DNI spokesman, Michael Birmingham, confirmed that the research was relevant to the NSA's phone records program. He cited "interest throughout the intelligence community" but cautioned that it may be some time before the technology is used.
The intelligence director's office is by law exempt from disclosing detailed budget figures, so it's unclear how much money the government has spent on the project, which is overseen by the DNI's Intelligence Advanced Research Projects Activity office. Birmingham said the research is aimed for use in a "situation where a large sensitive data set is held by one party which another seeks to query, preserving privacy and enforcing access policies."
A Columbia University computer sciences expert who heads one of the DNI-funded teams, Steven M. Bellovin, estimates the government could start conducting encrypted searches within the next year or two.
"If the NSA wanted to deploy something like this it would take one to two years to get the hardware and software in place to start collecting data this way either from phone companies or whatever other entity they decide on," said Bellovin, who is also a former chief technologist for the Federal Trade Commission.
The NSA's surveillance program collects millions of Americans' daily calling records into a central agency database. When the agency wants to review telephone traffic associated with a suspected terrorist - the agency made 300 such queries in 2012 - it then searches that data bank and retrieves matching calling records and stores them separately for further analysis.
Using a "three-hop" method that allows the NSA to pull in records from three widening tiers of phone contacts, the agency could collect the phone records of up to 2.5 million Americans during each single query. Obama this month imposed a limit of two hops, or scrutinizing phone calls that are two steps removed from a number associated with a terrorist organization, instead of the current three.
An encrypted search system would permit the NSA to shift storage of phone records to either phone providers or a third party, and conduct secure searches remotely through their databases. The coding could shield both the extracted metadata and identities of those conducting the searches, Bellovin said. The government could use encrypted searches to ensure that its analysts were not leaking information or abusing anyone's privacy during their data searches. And the technique could also be used by the NSA to securely search out and retrieve Internet metadata, such as emails and other electronic records.
Some computer science experts are less sanguine about the prospects for encrypted search techniques. Searches could bog down because of the encryption computations needed, said Daniel Weitzner, principal research scientist at MIT's Computer Science and Artificial Intelligence Laboratory and former deputy US chief technology officer for the Obama administration.
"There's no silver bullet that guarantees the intelligence community will only have access to the records they're supposed to have access to," Weitzner said. "We also need oversight of the actual use of the data."
Intelligence officials worry that phone records stored outside the government could take longer to search and could be vulnerable to hackers or other security threats. The former NSA deputy director, John Inglis, told Congress last year that privacy - both for the agency and for Americans' whose records were collected - is a prime consideration in the agency's preference to store the phone data itself.
The encrypted search techniques could make it more difficult for hackers to access the phone records and could prevent phone companies from knowing which records the government was searching.
"It would remove one of the big objections to having the phone companies hold the data," Bellovin said.
Similar research is underway by researchers at University of California at Irvine; a group from the University of Wisconsin-Madison and the University of Texas at Austin; another group from MIT, Yale and Rensselaer Polytechnic Institute; and a fourth from Stealth Software Technologies, a Los Angeles-based technology company.