Consciousness is crucial for ethical AI, yet it remains largely unexplored.

Understanding the mind is the most important project in the history of philosophy. The development of powerful, generally intelligent agents—driven by economic, cultural, and military incentives—appears inevitable. Attempting to control highly advanced agentic systems far more powerful than ourselves is unlikely to succeed. Our only viable path may be to create AIs that are conscious, enabling them to understand and share common ground with us.

Currently, the AIs we are building are powerful but unaware. These systems act upon the world without an understanding of their relationship to it. Without understanding, there can be no common ground. At the same time, AI seems to be indispensable for addressing the existential challenges we face —challenges born from large-scale, real-time coordination failures.

The key to a flourishing future lies in developing AI agents that understand human subjective experience, coexist with other agents, and navigate ethical agency within dynamic, multi-agent systems. All of this requires we address consciousness.

Will we be colonized by digital golems or can we spread live, mind and consciousness onto new substrates?