With tensions high along the Gaza border, Israel said its aircraft attacked 10 sites in the Palestinian enclave in response to persistent rocket strikes on southern Israeli towns.
But Prime Minister Benjamin Netanyahu signalled that broader Israeli action was not imminent.
The security source gave no details about the suspects arrested in the investigation into the abduction and killing of 16-year-old Mohammed Abu Khudair, other than to say they were Jewish and that police saw "nationalist motives" in the case.
There was no immediate announcement from police about the arrests.
Palestinian protesters throw stones towards Israeli forces (unseen) during clashes in Shuafat, East Jerusalem (AFP Photo)
Read: Israel frees jailed Palestinian-American boy, places him under house arrest
Abu Khudair's burnt body was discovered in a Jerusalem forest on Wednesday. Palestinians believe he was the victim of far-right Jews avenging the kidnap-killing of three Israeli teenagers who disappeared while hitchhiking in the occupied West Bank on June 12 and whose bodies were found on Monday.
Israel blames the deaths of Naftali Fraenkel and Gil-Ad Shaer, both 16, and 19-year-old Eyal Yifrah, on Hamas. The Gaza flare-up began in mid-June during Israel's search for the teens in the West Bank and its arrests of Hamas members.
Hamas has neither confirmed nor denied involvement in the killings.
In remarks to his cabinet on Sunday, Netanyahu pledged "to do whatever is necessary" to restore quiet to southern Israeli communities that have come under rocket attack from the Gaza Strip, where Hamas is the dominant force.
Mohammed Abu Khudair, 16, is seen in this undated family handout picture. Israel has arrested suspects in the death of Palestinian teenager Mohammed Abu Khudair, Israeli media reported on Sunday, and a security source indicated that Jewish assailants were responsible for the abduction and killing. (Reuters)
But he also cautioned against any rush toward wider confrontation with the group, whose arsenal includes long-range rockets that can reach Israel's heartland and its business capital, Tel Aviv.
"Experience has shown that during moments like these, one must act in a level-headed and responsible manner and not hastily," Netanyahu told his cabinet, in broadcast remarks.
Read: Slain Palestinian teen 'burned alive', autopsy shows; protests on
Israel beefed up ground forces along the Gaza frontier on Thursday, a deployment widely seen as sending a message to Palestinian militant groups that an invasion was an option if rocket fire did not stop.
Egypt, a bordering country, has been trying to mediate a truce.
But in a move that could harden militants' resolve to confront Israel, the military said it cut the fishing zone for Palestinian fishing boats off Gaza from six to three nautical miles.
The remains of a car torched by Israeli Arab rioters lies in a fiery street near Qalansawe, central Israel (AP Photo)
Israel releases Palestinian-American
East Jerusalem and other flashpoint Arab communities were quiet on Sunday after protesters angered by Abu Khudair's death clashed with police late on Saturday, a police spokesman said.
Palestinian-American Tariq Khdeir, 15, a cousin of the slain Palestinian teenager, was arrested by Israeli border policemen, who his family said beat him severely, during a protest on Thursday in East Jerusalem.
An Israeli court on Sunday ordered Khdeir released from jail and placed under house arrest for nine days. His mother said the family planned to return to Tampa, Florida on July 16.
The United States has called for an investigation, and Israel's Justice Ministry said the police internal affairs office was looking into the incident.
In Ramallah, Palestinian President Mahmoud Abbas said he sent a message to U.N. Secretary-General Ban Ki-moon urging the formation of an international investigation committee into "terror crimes conducted against our people, including the burning alive of Mohammed Abu Khudair".
An autopsy attended by a Palestinian pathologist showed soot in the teenager's lungs, indicating he was still alive when he was set on fire, the Palestinian attorney general said.
Read: Palestinian teen murder sparks rioting in Jerusalem