The bodies of all four children were found in various canals in Jind and Hisar districts of Haryana.
The police on Saturday arrested Rameshwar. The police said Rameshwar was unable to support his family financially.
He had an illicit relationship with a woman and used to spend his money on her, the police claimed.
Investigations revealed that in order to get rid of the children, Rameshwar stage-managed their kidnapping and after murdering them, threw them in the canals.