FIFA should limit terms for senior officials, set up an independent group to clear up anti-corruption allegations and "embrace transparency", corruption watchdog Transparency International (TI) said on Tuesday.
TI said that, despite recent measures, world soccer's governing body still gave the impression of being run "like an old boys' network".
It added that the new group should be composed of representatives from outside FIFA, such as elder statesmen, sponsors, media and civil society, and inside football, such as players, women's football, referees and supporters.
"FIFA says it wants to reform, but successive bribery scandals have left public trust in it at an all time low," said Sylvia Schenk, TI's senior advisor on sport.
"Working with an oversight group -- taking its advice, giving it access, letting it participate in investigations -- will show whether there is going to be real change. The process has to start now."
FIFA president Sepp Blatter, re-elected for a fourth term in June, promised to create a new committee to act as a watchdog, mentioning former US secretary of state Henry Kissinger and Spanish tenor Placido Domingo as possible members.
He has yet to announce further details.
In its report, the Berlin-based watchdog, which issues a global league table of the least corrupt nations, said: "FIFA is both a non-governmental, non-profit organisation and a global company with huge revenues, unprecedented reach, political clout and enormous worldwide social influence."
But it said FIFA was only accountable to its 208 member associations who elected the FIFA president and in turn, received handouts from soccer's governing body.
"This lack of mandatory accountability to the outside world makes it unlikely that change will come either from within the organisation or from the grassroots of the football organisations," the report added.
FIFA has been hit by a series of corruption scandals in the last year.
Two executive committee members were banned last November for allegedly offering to sell their votes in the contest to host the 2018 and 2022 World Cups, subsequently awarded to Russia and Qatar respectively.
Last month, Asian Football Confederation chief Mohammed Bin Hammam was banned for life for allegedly trying to buy votes in the June presidential election where he was a candidate.
Fellow executive committee member Jack Warner, a leading FIFA powerbroker, quit after being put under investigation in the same case.
All decisions were taken by FIFA's ethics committee but TI said this did not go far enough.
"The lack of a fully transparent investigation leaves the root of the problem untouched," it said.
"Similar scandals have hit FIFA before, and without a comprehensive process that deals with all allegations from the past, with consequences for anyone who has behaved unethically... and/or given or requested bribes, the scandals are likely to recur."
It said the ethics committee hearing took place behind closed doors and added: "The members of the ethics committee are appointed by FIFA's executive committee, raising doubts about their independence, and not all allegations discussed in the public have yet been dealt with."
Similarly, TI said FIFA needed to do more to improve its reporting and accounting standards.
"The officials leading world football still give the impression of operating as an "old boy's network".
"With only three presidents since 1961 and the current president in his fourth term, FIFA does not match standards for rotation of top positions set by businesses or by other large organisations."
Positions on the executive and finance committee, currently unlimited, should be limited to two terms.
Listing a number of suggested improvements, TI said: "A new era for FIFA requires a review of its internal governance and the introduction of transparency and accountability into its decision-making processes and operations.
"This is a critical step that FIFA must take if it is to become a sustainable, accountable and transparent organisation."
The report concluded: "Throughout its history, the workings of football's governing body have been opaque. However, people across the world, in all walks of life, are calling for an end to "business as usual" and demanding accountability from those in power.
"If FIFA wants to rebuild trust it must embrace transparency.
"Football's governing body must be an example of the fair play that it promotes on the pitch."