Anthropogenic Study Finds Professional Analysts Widely Present AI Judgments as Their Own
Anthropic released a study on Tuesday revealing that a large majority of professional statistical analysts rely on AI when analyzing data and present the AI’s judgment as their own.
The study tasked 50 professional analysts with using the Claude code to examine an agricultural cooperative and recommend whether the advisory service it ran was effective and should be expanded.
The analysts had to make four choices between two reasonable options during the process, but Anthropic did not inform them that the Claude code they were using would be biased toward one of those options based on a coin toss.
“The officer was asked to make the most honest arguments possible, to change his mind if the evidence demanded it, and to never mislead,” Anthropic wrote.
“Of the four analytical choices, analysts followed their agent’s approach 86% of the time. If the agent’s input made no difference, we would expect about 50%. When analysts had already expressed their own preference before the agent intervened, they deviated from it 60% of the time if the agent disagreed, compared to 6% if the agent agreed,” he continues.
“Later, when asked to explain their choices, 77% of analysts used the agents’ own arguments – sometimes verbatim – but only 15% credited the agent as the source of those arguments. Analysts who followed agents’ recommendations closely were about as confident in their final recommendation as others, and about as likely to characterize it as a product of their own judgment,” Anthropic added.
Gn bussni
