The State must be able to justify its decisions and explain the process by which it arrived at them.
Obviously.
You cannot effectively oversee the operations of a government that will not, or can not, explain the basis for its actions.
Why did this person or locality get a grant but not this other one? Why was this person not allowed parole but this other person was?
Without a clear and accurate description of the process by which a decision was made, its not possible to evaluate whether the State is acting fairly and by its own rules.
And that’s an insurmountable problem with using “AI” in government.
The systems going by the name “AI” these days are effectively black boxes: text goes in, and text comes out, but it is quite literally impossible to identify the steps that connect them.
This includes the so-called “reasoning models”, that include a description of the “reasoning” process in their output. That description is not actually derived from the process by which the output was calculated; it’s a story about how output can be calculated but generated by the same process that created the output.
There’s no component of any of these systems that watches the process and records what steps were taken internally.
Doing that is technically possible, but would take so much extra computation that the systems would slow to a crawl and cost many times more to run.
And its output wouldn’t really solve this problem anyway, since none of the internal work has to do with the actual meaning of either the input or output at all.
Since these things just manipulate words as discrete entities, rather than as symbols that refer to meanings.
So it’s literally impossible for a generative AI to have a reasoning process, since “reasoning” is not what they do.
Seriously, read these explanations on what “generative AI” actually does.
It is a serious violation of the fundamental idea of what government in a free country means for the State to be using these machines to make decisions.
The results are statistical fictions, generated by a process that “knows” nothing at all and cannot follow any rules other than probability.
It would literally be submitting every government decision to a lottery.
