The CISF is investigating complaints that its men forced some women entering the Taj Mahal to take off their stoles bearing the words 'Ram Ram'.
Commandant AP Singh of the Central Industrial Security Force said an inquiry was on and the guards posted at the Taj that day would be punished if found guilty.
Both Singh and the Archaeological Survey of India's NK Pathak also said that the matter had been sorted out.
They said a "consensus" had been reached at a meeting also attended by Agra's district magistrate and the CISF inspector general last week.
But BJP MP Ram Shankar Katheria hit out at the CISF.
"CISF personnel went beyond their brief," Katheria told IANS. "When there is no dress code, why were the women humiliated and forced to surrender their dupattas?"
"At monuments like the Taj, you need a force that is cooperative, decent and clean. The force must respect the culture of visitors and show due courtesy. I am determined to take this issue to its logical end," he said.
According to officials, nearly six million people visit the Taj annually.