the killing of 85-year-old VHP leader Swami Laxmananda Saraswati in August last year.
"I am sorry that certain things happened last year and you have been brought to these camps. But you must go back to your villages. I am here to tell you, don't fear," he said.
Condemning the incident, Chidambaram asked them to start life afresh, build the churches and practice their religion.
"Whatever happened was wrong. Build your churches and practice your dharma," he said.
When some refugees spoke of fear of RSS and Bajrang Dal, the Home Minister assured then that the guilty would be "prosecuted and punished."
"All those who have indulged in this ghastly crime will be prosecuted and punished," he said.
The Home Minister visited Mandaika, Rahikola, Tinigia and Tikabali camps in the district providing shelter to about 1500 people.
The Home Minister, who is on his first visit to Orissa after the Lok Sabha polls, patiently listened to the plight of the victims living the camps.
"Bringing back all displaced is our endeavour. What happened was wrong. Over 4,620 families were affected out of which majority have returned. Now only a few -- approximately 1,500 -- have to go back," he said.
On some residents' complaint that the state government's relief of Rs10, 000 was inadequate, Chidambaram assured them that he would take up the matter during his meeting with Chief Minister Naveen Patnaik.
"Christians are living all over the country and it is not correct to say they are not being protected," he said.
The Kandhamal riot victims are being used by Naxals for creating communal divide in the area in order to strengthen their base among the local minority population, the officials told the Home Minister.
Several groups of Naxals have been seen tying up with minority groups in the area to win their support, official sources said.
"The killing of Saraswati was the handiwork of Naxals. The RSS leaders, who were arrested during the riots, were also killed by them in a targetted manner after their release," they added.