Sympathy for the Machine | Curious Archive

The subject deserves a better treatment than this relatively shallow pop-scifi video. This has been a question in science-fiction for more than a century. No, it did not start with Star Trek, Asimov dates it as far back as 1818 Frankenstein. If you are bold you can see this theme in the 2000 years old story of Talos, the bronze colossus that wanted immortality (ancient Greece was surprisingly full of automatons, Rhodes was known for them).

The question is what does imbue humans with what makes us see them as humans? Please don't use the word "soul". It is meaningless and religious, does not refer to any observable thing.

And don't use "intelligence" as an interchangeable word with that undefined property, that most sci-fi authors have took to call "sentience". That word is not human-centric and they typically apply sentience or the question of sentience to aliens or machines.

We have a hard time seeing as sentient something that has zero sense of ego. You can make an extremely intelligent machine with no ego, no sense of self. This is what you have in LLMs.

Giving them a sense of self and ego is probably feasible, but it is both useless and a huge responsibility. Maybe will happen first as an art project, but then you have to question the morality of creating something that does not want to die (or at least expresses it) but is not recognized as a person.

It also interrogates our notion of the linearity of the self. If such a sentient being can be forked, suspended, copied, have memories wiped out, fake memories implanted, personality changed, willingly or unwillingly, that opens a lot of philosophical questions.

I wish the community would embrace them, but so far all we have had are extremely superficial debates over "true" intelligence, usually defined as the difference between what humans can do and machines can do, an ever-shrinking territory.