Thomas Metzinger (What to think about machines that think)

Thomas Metzinger explores the concept of suffering in the context of artificial intelligence and consciousness. Key points include:

1. Efficiency and Suffering: Metzinger suggests that human thinking is efficient because humans experience suffering. Suffering is intertwined with high-level cognition and intrinsic motivation. It arises from the fragility of human bodies, the challenges of social environments, and the constant struggle against mortality.

2. AI and Suffering: Metzinger raises questions about whether artificial intelligence (AI) will also need to be associated with suffering to be considered intelligent. He questions whether suffering is a necessary feature of intelligent systems or merely a contingent outcome of evolution.

3. Conditions for Suffering: Metzinger outlines four necessary conditions for suffering in conscious systems:

– C (Consciousness) Condition: Only beings with conscious experience can suffer.

– PSM (Phenomenal Self-Model) Condition: Suffering presupposes self-consciousness, specifically the possession of a phenomenal self-model.

– NV (Negative Valence) Condition: Suffering is created by integrating states representing negative value into the self-model.

– T (Transparency) Condition: Transparent phenomenal states make their content appear irrevocably real.

4. Ethical Consideration: Metzinger argues that any system satisfying all four conditions should be treated as an object of ethical consideration, as it might have the capacity to suffer. Ethical caution should be exercised when dealing with such systems.

5. Possibilities of AI: Metzinger explores four possibilities regarding AI and suffering:

– Unconscious robots cannot suffer.

– Conscious robots without a coherent self-model cannot suffer.

– Self-conscious robots without negatively valenced states cannot suffer.

– Conscious robots without transparent phenomenal states cannot suffer.

Metzinger concludes by pondering whether real intelligence can exist without an existential concern and whether AI systems could be designed to be superbly intelligent but incapable of suffering.

YARPP List

Related posts:

1. Law 17: Seize the Historical Moment (The Laws of Human Nature)
2. Part 2: Isolate the Victim (The Art of Seduction)
3. Chapter 16: The Capitalist Creed (Sapiens)
4. On Nietzsche’s Thus Spoke Zarathustra Summary (8.4/10)