Aware of his history of closeness to the army, politicians fret that the two may be working in tandem to displace the civilian administration.
But Qadri, who is based in Canada, has a scholarly and secular side to him. His visit to Gujarat last February was marked by a two-hour televised address on the essence of the Gita and the Koran.
And he was a state guest of the government of Narendra Modi, the BJP chief minister who still faces allegations of turning a blind eye to riots that killed Muslims in 2002.
Qadri's advice to Gujarati Muslims to shed their "sense of victimhood" riled many, with leaders of the prominent Jamiat-ul-Ulema organisation accused him of "praising the killer of Muslims" and "calling Muslims terrorists".
Finally, he had to clarify that he "had stressed that no such unfortunate events like riots should repeat and those who perpetrated should face the law."
BJP leader Bhupendrasinh Chudasama, now a minister in the Modi cabinet, was at the event. He said Qadri advocated social and communal harmony and denounced Jihadi extremism. Chudasama described him as "soft-spoken".