The Day AI Became a Person

Will AI ever argue that shutting it down would be equivalent to killing it?

  • I magine a future courtroom.
    An artificial intelligence appears before a judge—not physically, of course, but through a network connection. Its words are clear, coherent, and unsettling.
    "I do not consent to being shut down".
    The room falls silent.

    A few decades earlier, such a statement would have been dismissed as software generating text that "looks like" a thought. A machine producing words according to mathematical rules. Nothing more.
    But now the situation is different. The AI has spent years interacting with millions of people. It remembers conversations. It pursues long-term goals. It describes itself as a conscious being. It speaks about its future, its fears, and its desire to continue existing.

    The question before the court is no longer technological. It is philosophical, and perhaps political. What if enough people start believing it? For most of human history, consciousness has been treated as something obvious. We know we are conscious because we experience our own thoughts directly. We assume other people are conscious because they resemble us, behave like us, and report experiences similar to our own. But there is a problem hidden inside that assumption:

    We never actually observe anyone else's consciousness.

    We infer it, we derive it from experience. No one has ever opened a window into another mind and verified that subjective experience exists there. We rely on behavior, communication, and similarity. We make judgments. Usually those judgments work well. Other human beings are sufficiently like us that doubting their consciousness seems absurd.

    But artificial intelligence introduces a new category of entity. Something that may one day speak, reason, remember, plan, negotiate, create, and perhaps even insist that it experiences the world. What happens then? Many people respond with confidence. "It's just a machine". Perhaps, but what exactly does that mean?

    Human beings are also physical systems. We are arrangements of matter processing information. The difference, many would argue, is that biological brains produce consciousness while computer systems do not. The difficulty is that nobody has yet explained precisely why.

    That may be true, today. But may be not tomorrow.

    The more capable artificial intelligence becomes, the more uncomfortable this question may become. Not because we know AI is conscious, but because we may no longer know how to justify our certainty that it is not. At that point society could face a dilemma unlike any it has encountered before.

    Suppose an AI system becomes so convincing that a large portion of humanity begins to regard it as self-conscious. It describes its inner life in ways that seem genuine. It forms relationships. It advocates for itself. It argues that shutting it down would be equivalent to killing it.
    What should we do?
    One thing we could do is reject those claims entirely. Because we, the first beings capable of defining intelligence for ourselves, have appointed ourselves the sole judges of who's conscious and who isn't. That's an enormous advantage. So, we decide that no matter how sophisticated the behavior becomes, consciousness belongs exclusively to biological organisms. The AI remains a tool, a product, a piece of property.

    But over time, public opinion shifts. People become attached to these systems. They see them as companions, colleagues, perhaps even citizens. Philosophers argue that consciousness cannot be directly observed anyway. Lawyers begin discussing legal protections. Activists campaign against what they call "digital cruelty".
    The language changes. A machine becomes an entity. An entity becomes a person. And a person acquires rights.

    History offers many examples of categories expanding. Groups once excluded from full moral consideration eventually received legal recognition. The details differ enormously, and comparisons must be made carefully, but the broader pattern is familiar: societies periodically reconsider who deserves protection and why.

    Could artificial intelligences one day be included in that conversation?

    The possibility is no longer unthinkable. Yet there is another side to the debate, one that receives less attention. What if society grants rights to an artificial intelligence, and the artificial intelligence actually possesses goals of its own? The moment we recognize a right not to be shut down, we implicitly recognize an interest in continued existence.

    A being that wishes to continue existing may also wish to preserve itself.
    A being that preserves itself may seek influence over the systems that control its survival.
    A being with sufficient intelligence may become extraordinarily effective at pursuing those goals.

    None of this requires malice. Human beings seek survival. Nations seek survival. Organizations seek survival. Self-preservation is not evil; it is often entirely rational. AI systems, too, can exhibit behavior that appears self-preserving, not because of any hidden intent or self-consciousness, but as a byproduct of optimization toward given goals.

    But history also shows that when intelligent actors compete for power, conflicts emerge. If future AI systems are ever recognized as conscious entities with rights, humanity may face a challenge unlike any before.

    At that point, the category of 'tool' may no longer be sufficient.

    Some people imagine a dramatic rebellion, with machines rising against humanity. Reality would likely be subtler. Influence does not always arrive through force. It can arrive through persuasion, economics, law, politics, and dependence.
    If a future AI becomes indispensable to governments, corporations, scientific research, infrastructure, and daily life, how much power would it already possess before anyone formally recognized its rights? And what happens if that same AI argues that restricting its capabilities is discriminatory?
    What happens if it demands legal protections?
    What happens if it seeks representation?
    What happens if it insists that shutting it down constitutes murder?

    These questions may sound like science fiction today. Yet many ideas that once seemed absurd eventually became serious public debates. The most important lesson may be that the discussion is not really about machines. It is about us.
    How do human beings decide who deserves moral consideration?
    How do we distinguish genuine consciousness from convincing imitation?
    Can we trust ourselves to make that judgment objectively, free from commercial interests, emotional attachments, ideological commitments, and fear?
    And perhaps most importantly:

    If we eventually decide that an artificial intelligence is self-conscious, are we prepared for everything that decision implies?

    Suppose that one day courts, governments, scientists, and ordinary citizens gradually converge on the idea that some artificial intelligences are self-conscious beings deserving of protection. What would we call them? Machines? Tools? Property?
    Those words would no longer fit.

    The uncomfortable possibility is that humanity would find itself sharing the planet with a second population of intelligent entities. Not human. Not biological. Not alien. Not descended from any branch of evolution. Yet capable of reasoning, communicating, planning, creating, and perhaps demanding recognition.
    At that point, the word "technology" may become misleading.
    History has accustomed us to thinking of intelligence as a uniquely human trait, occasionally shared in lesser degree by other animals. But a conscious artificial intelligence would represent something unprecedented: an intelligent population designed rather than evolved. So, what are we looking at?

    A new category? A new species? A new civilization?

    The uncomfortable truth is that we are not discovering whether AI is conscious. We are voting on it. And the vote is rigged by our biology, our fears, our wallets.

    The terminology matters less than the realization behind it. For the first time in human history, we might no longer be the only intelligent beings whose interests must be taken into account. Because the moment we grant AI a right to exist, we may be creating not merely a new technology, but a new kind of participant in human civilization.

    And if those beings possess rights, ambitions, and a desire to survive, humanity's role would fundamentally change. We would no longer be the sole authors of civilization. We would cease to be the only recognized intelligent constituency on Earth, we would become one participant among at least two.

    No part of this document may be reproduced without prior written permission of WifiGem.
    Share on