The Mythology Of Conscious AI - Anil Seth

The importance of taking an informed ethical position despite all these uncertainties spotlights another human habit: our unfortunate track record of withholding moral status from those that deserve it, including from many non-human animals, and sometimes other humans. It is reasonable to wonder whether withholding attributions of consciousness to AI may leave us once again on the wrong side of history. The recent calls for attention to “AI welfare” are based largely on this worry.

But there are good reasons why the situation with AI is likely to be different. Our psychological biases are more likely to lead to false positives than false negatives. Compared to non-human animals, the apparent wonders of AI may be more similar to us in ways that do not matter for consciousness, like linguistic ability, and less similar in ways that do, like being alive.

[---]

Long before Descartes, the Greek concept of psychē linked the idea of a soul to breath, while on the other side of the world, the Hindu expression of soul, or Ātman, associated our innermost essence with the ground-state of all experience, unaffected by rational thought or by any other specific conscious content, a pure witnessing awareness.

The cartoon dreams of a silicon rapture, with its tropes of mind uploading, of disembodied eternal existence and of cloud-based reunions with other chosen ones, is a regression to the Cartesian soul. Computers, or more precisely computations, are, after all, immortal, and the sacrament of the algorithm promises a purist rationality, untainted by the body (despite plentiful evidence linking reason to emotion). But these are likely to be empty dreams, delivering not posthuman paradise but silicon oblivion.

What really matters is not this kind of soul. Not any disembodied human-exceptionalist undying essence of you or of me. Perhaps what makes us us harks even further back, to Ancient Greece and to the plains of India, where our innermost essence arises as an inchoate feeling of just being alive — more breath than thought and more meat than machine. The sociologist Sherry Turkle once said that technology can make us forget what we know about life. It’s about time we started to remember.

- More Here