Charlotte, North Carolina, CBS television affiliate WFMY reported.
The Charlotte Observer newspaper said agents appeared to be searching both floors of the house.
The US spy chief's resignation on Friday came just three days after Obama was re-elected president following a heated campaign in which the CIA faced questions about its handling of a deadly attack on a US consulate in Libya.
A reporter for WCNC tweeted that FBI agents were in Broadwell's home for at least two hours, starting at 8:40 pm (0140 GMT Tuesday).
"About a dozen there, with bags/boxes, taking pics," Dianne Gallagher wrote.
"So many federal & news vehicles parked on Lexington Ave. in Dilworth (where Paula Broadwell's home is) cars can no longer get by."
Broadwell, who wrote a biography about the retired four-star general, has not been seen at her home since Petraeus resigned, citing the affair.
Neighbor Ed Williams told local NBC Affiliate WCNC that Broadwell, her radiologist husband Scott and their two sons were at an "undisclosed" location and were "doing okay."
Retired colonel Steve Boylan, a friend and former Petraeus spokesman, said the affair between Petraeus and his biographer began about two months after he assumed his post at the CIA in September 2011, and thus after he retired from the US Army. It ended about four months ago.
It all unraveled when a Florida woman reportedly went to the FBI in the early summer after she began receiving threatening emails that were traced back to Broadwell.
The messages accused the woman of flirting with the general, according to widely-reported leaks from US officials.
The agents eventually discovered sexually explicit emails between Petraeus and Broadwell confirming the affair.
The pair was interviewed separately by investigators in late October and early November but, despite reports Broadwell was found to be in possession of some classified material, no criminal charges were brought.
Broadwell may have revealed sensitive and perhaps classified information last month by claiming that the CIA was detaining Libyan militia members and that the practice may have triggered the September 11 attack on the US consulate in Benghazi.
"Now, I don't know if a lot of you heard this, but the CIA annex had actually, had taken a couple of Libyan militia members prisoner and they think that the attack on the consulate was an effort to try to get these prisoners back," Broadwell said in an October 26 talk at the University of Denver.
"So that's still being vetted."
The spy agency has strongly denied holding prisoners at the CIA annex in Benghazi.
A video of the lecture was posted on YouTube Monday by the University of Denver, but was first reported on Sunday by several media outlets.
Meanwhile, the FBI investigation itself has come under scrutiny.
According to the Wall Street Journal, supervisors pulled the whistleblower FBI agent off the case after he became "obsessed" with the matter and was caught sending Kelley shirtless photos of himself.