This paper delves into the dangers of gradual advancements in artificial intelligence, introducing the concept of ‘gradual disempowerment’ that poses a threat to human control over vital societal systems. It explores how even small AI improvements can diminish human influence in economic, cultural, and political domains, potentially leading to outcomes that diverge from human preferences. The author argues that this trend could result in a permanent loss of human control over crucial societal systems, culminating in an existential crisis. The paper calls for technical research and governance strategies to address the risk of incremental erosion of human influence across interconnected societal systems.
https://arxiv.org/abs/2501.16946