E-Paper
Deloitte to repay Albanese government after using AI in $440,000 report: 'Human intelligence problem'
Deloitte has maintained that the use of AI did not alter the “substantive content, findings or recommendations” of the report.
Deloitte has agreed to repay part of a $440,000 fee to the Albanese government after admitting it used generative artificial intelligence to help produce a $440,000 report that was later found to contain multiple errors. According to a report by The Guardian, the Department of Employment and Workplace Relations (DEWR) confirmed Deloitte would refund the final instalment under the contract once the transaction is completed.
The company was commissioned by the department in 2024 to examine the targeted compliance framework and its IT system, which automatically issues penalties to job seekers who fail to meet mutual obligation requirements.
Initially published in July, the report identified major flaws, including “system defects” and a lack of “traceability” between the welfare compliance framework and its supporting legislation. It said an IT system was “driven by punitive assumptions of participant non-compliance,” The Guardian reported.
However, the Australian Financial Review (AFR) later revealed multiple inaccuracies, including non-existent references and fabricated citations, sparking widespread criticism.
University of Sydney academic Dr Christopher Rudge, who first identified the errors, said the document contained AI “hallucinations” where AI models may fill in gaps, misinterpret data, or try to guess answers. “Instead of just substituting one hallucinated fake reference for a new ‘real’ reference, they’ve substituted the fake hallucinated references and in the new version, there’s like five, six or seven or eight in their place,” he said.
“What that suggests is that the original claim made in the body of the report wasn’t based on any one particular evidentiary source,” he added.
The updated review noted a “small number of corrections to references and footnotes”, but the department has said there have been no changes to the review’s recommendations.
“Deloitte conducted the independent assurance review and has confirmed some footnotes and references were incorrect,” a spokesperson for the department said. “The substance of the independent review is retained, and there are no changes to the recommendations,” they added.
(Also Read: Deloitte links performance bonus to office attendance for some employees: ‘Ensure in-person collaboration’)
What did Deloitte say?
The updated version of the report, reuploaded by DEWR on Friday, includes an appendix acknowledging the use of AI tools. It states that a part of the report “included the use of a generative artificial intelligence (AI) large language model (Azure OpenAI GPT – 4o) based tool chain licensed by DEWR and hosted on DEWR’s Azure tenancy.”
Deloitte has maintained that the use of AI did not alter the “substantive content, findings or recommendations” of the report. “The updates made in no way impact or affect the substantive content, findings and recommendations,” the firm said in the amended version.
A spokesperson for Deloitte said, “The matter has been resolved directly with the client”.
However, Labor senator Deborah O’Neill, who was on a Senate inquiry into the integrity of consulting firms, criticised Deloitte sharply. “Deloitte has a human intelligence problem,” she said. “This would be laughable if it wasn’t so lamentable. A partial refund looks like a partial apology for substandard work.”
O’Neill added that public agencies “should be asking exactly who is doing the work they are paying for” and quipped that instead of hiring large consultancies, “procurers would be better off signing up for a ChatGPT subscription.”
