The NSW Ombudsman has called for greater transparency around Government Agencies’ use of artificial intelligence (AI) and plans to map its use in administrative decision-making processes across the State.
In his Report, The new machinery of government: using machine technology in administrative decision-making, the Ombudsman, Paul Miller cautioned Agencies that using machine technology in ways that did not accord with standards of lawfulness, transparency, fairness and accountability could lead to findings of maladministration or potentially unlawful conduct.
Mr Miller said Agencies were not obligated to proactively report on their use of machine technology and did not routinely inform people when decisions affecting them were being made by or with the assistance of machines.
“As an integrity Agency, our concern is that Agencies act in ways that are lawful, that decisions are made reasonably and transparently, and that individuals are treated fairly,” he said.
“Those requirements don’t go away when machines are being used.
“The NSW Ombudsman’s Office was prompted to write the Report after becoming aware that the State’s debt-collection agency, Revenue NSW, was using an automated technology system to garnishee, or debit money from, the bank accounts of people who had failed to pay fines.”
Mr Miller said that, following his Office’s intervention, Revenue NSW took steps to address concerns about unfairness, however, did not seek expert legal advice on whether the use of the automation process was lawful and in accordance with its powers under the Fines Act 1996.
The Ombudsman said his Office chose to seek that legal advice, which concluded that the original automated process used by Revenue NSW to garnishee bank accounts was unlawful.
He said that, in response to his Office’s initial concerns, Revenue NSW modified its process in 2019 so that orders to take money from bank accounts were not fully automated.
“We are concerned that other Agencies may also be designing and implementing machine technologies without appreciating all the risks, without transparency, and without getting appropriate legal advice,” Mr Miller said.
The Ombudsman said his Report provided guidance to Agencies on the steps they should take to reduce the risk that machine technology could be unlawful or otherwise amount to maladministration.
He said this included ensuring the design involved experts from fields other than IT, including legal advisors; giving careful consideration to the relationship between the machine and the ultimate human decision-maker; building in a rigorous pre-deployment testing and ongoing auditing regime; and taking action to ensure appropriate transparency.
The Report also suggested that Government Agencies seek Parliamentary approval through legislation before technology was adopted for important administrative functions.
Mr Miller said his Office would engage with the NSW Government and Local Government sector to comprehensively map the use of machine technology in administrative decision-making processes across the State.
The Ombudsman’s 97-page Report can be accessed at this PS News link.
