Automation Bias

Junior developers' over-reliance on AI-generated code has extended the development cycle for their debugging skills and systems design thinking.

Automation bias refers to the gradual degradation of critical thinking that occurs when humans are consistently placed in the role of confirming algorithmic conclusions.

Automation bias is the psychological phenomenon in which humans tend to over-trust the output of automated systems, and was first extensively studied in the fields of aviation and medicine.

Compliance takes the approach of AI-assisted screening with human final decision-making, rather than replacing human reviewers entirely with AI.

Automation bias refers to the tendency of people to progressively reduce their scrutiny and questioning of system outputs when working with automated systems over time, and was first systematically studied in the aviation field.

The term "automation bias" was originally proposed by psychologists Linda Skitka and Kathleen Mosier in 1999 while studying aviation decision-making systems, referring to the human tendency to uncritically accept suggestions from automated systems

Automation Bias was first systematically described in aviation automation research, with studies in the 1980s finding that pilots often tended to follow the autopilot system even when faced with obviously erroneous automated decisions.

心理学将人们对AI输出的过度信任归因于自动化偏差（Automation Bias）

Blindly trusting AI output and using it directly without review is a phenomenon that warrants caution.

Automation bias peaks at the moment when a developer runs AI-generated code with basic inputs, sees it work, and then stops testing on the assumption that 'the model has already handled all the other cases.'

Translations generated by large language models read fluently even when they drop clauses, lose negations, round numbers, or invent wording that changes the meaning, which is more dangerous than the obvious errors of statistical machine translation because reviewers are more likely to overlook errors in fluent text (automation bias/fluency heuristic).

Research on "automation bias" in cognitive psychology shows that when a system frequently pops up approval requests, human operators tend to click "approve" quickly without substantive review.

Automation bias was first systematically studied in the aviation field and was later repeatedly verified in fields such as medical diagnosis, military command, and financial trading.

Introducing automated review may remove the review layer originally provided by human experts, and organisations often cut staffing after automation, creating hard-to-reverse path dependence (automation bias and diffusion of responsibility).

在航空、医疗等领域，自动化偏见已被证明会导致严重判断失误。

认知错觉与自动化偏见是此处两个重要的心理学概念。

Evaluating automation bias requires specifically incorporating observation of the clinical decision-making process, for example recording the proportion of doctors who modify AI recommendations, the response time to AI prompts, and the frequency with which erroneous AI outputs are intercepted or let through, rather than focusing only on final outcome metrics.

Automation Bias is a psychological concept referring to the tendency of human decision-makers to reduce critical scrutiny when a system presents information in an authoritative and definitive tone.

There is extensive documentation in aviation showing that pilots who over-rely on autopilot experience degraded judgment when the system issues anomalous signals.

The high accuracy of large models on everyday tasks provides fertile ground for automation bias—the higher a system's accuracy rate, the more easily its errors are overlooked by humans.