The Green Triangle

Where is it?

Close your eyes and picture a green triangle.

Now, where is that green triangle? If I cut open your brain while you were thinking of it could I find that green triangle inside? Well, there obviously isn’t a green triangle lodged in between the folds of your brain. Fine, so what if I scanned your brain using an fMRI machine at that time? Would I see it in the scans of your brain? Probably not. What I’d probably see is the neuronal activity that corresponds to the visualisation.

But that word — corresponds — hides the real problem. There appears to be a difference between the physical mechanisms that allow for that visualisation and the subjective experience of the visualisation itself. The brain activity is public, measurable, and describable. The experience of ‘seeing’ the green triangle is private, unshareable, and, in some sense, not located anywhere in space.

This Explanatory Gap — the one between the measurable and the experienced — is the mystery.

Other Minds

If I asked you where ‘you’ are located in your body, you’d probably point to your head. That’s where the eyes, ears, and brain live. If I asked you the same question about your dog, you’d point to its face too. The same would happen for cats, chimps, the lot. We intuitively centralise other beings’ experience in their head.

But there are minds that don’t work like that at all. The most alien mind on this planet might already be here — swimming in our oceans. Let me introduce my favourite animal: the octopus.

Complex life has existed for ~630 million years. Our lineage split from the octopus near the very beginning of that timeline. So, we developed our nervous systems independently of one another.

Since we developed our minds and brains so differently, our neurology is also very different. About 95% of our neurons are in our brain and spine and the remaining are spread throughout our body. Juxtapose that with the octopus for whom about 70% of its neurons are not in its head, but spread across its tentacles.

The tentacles can each taste, touch, feel, even reason, and they can do this independently of one another.

The experience of the octopus would not be in its head, but across its whole body — head, beak, eight tentacles, shifting skin.

So what’s the point of this octopus example? Simply put,

Consciousness takes forms we could never imagine.
And minds appear to be shaped by the bodies that carry them.

So, would the octopus see the green triangle?

I don’t think so — it might taste geometric patterns in the water or feel shapes and objects with neurons that live in its arms. Its consciousness doesn’t live behind eyes that point forward but is instead distributed across a body that thinks with its skin.

The green triangle exists in my human consciousness. Something else —unnameable by us — exists in theirs.

This is an animal we can study and dissect. It’s one that we have been studying and dissecting, and one that we share the same building blocks of life with; and we have not been able to bridge the gap between our experience and its experience. We haven’t even come close.

So what happens when the mind in question isn’t even biological at all?

The Dream of AGI They Keep Selling

Artificial General Intelligence (AGI) is the next rung on the ladder after AI. Right now, it’s a dream — or, if we’re being charitable, a distant goal.

But what is it? One would think that the goal of the most well-funded research effort in the history of our civilisation would have a more precise definition.

The vagueness isn’t a bug, it’s a feature.

When AI couldn’t hold a conversation, AGI meant a machine that could talk. When AI learned to talk, AGI meant a machine that could reason. When AI started reasoning, AGI became something that truly understands. The finish line moves every time the runners get close. Nothing that has actually been built ever qualifies. Whatever AGI is, it’s always the next thing.

The lizards will tell you, ‘AGI is just around the corner’ or ‘We will have AGI by 2027’ the same way they told you, ‘We’ll have self-driving cars by 2020’ or ‘Humans on Mars by 2025’.

A precise definition of AGI would be too dangerous. It would mean that it can actually be tested. Which means it could be failed. A vague one can absorb any amount of progress without ever being satisfied, which makes it the perfect product: permanently almost-here.

Right now, it’s marketing narrative parading as a scientific milestone. And narratives don’t need falsifiability. They just need endless amounts of money to build data centres so that we can get (n-1) steps towards AGI.

To be fair, I don’t think most of the people actually building these systems are part of the con. The AI researchers and engineers — someone I hope to be one day — are doing extraordinary and genuine work. But the term AGI has been captured by the people who fund that work, and they need it to stay undefined. Precision would mean accountability. Vagueness means another funding round.

So, what if we had an honest definition of AGI?

Even if we could somehow take the term ‘AGI’ back from the marketers and give it a rigorous, honest, and testable definition, we would still be stuck.

Because the things AGI is supposed to produce — experience, awareness, consciousness, take your pick — don’t have real definitions either. Philosophers and neuroscientists have been working on this problem and have made no progress since humans first sat down around a fire and asked, ‘What am I?’. We’ve been at it for centuries. We don’t know what it is. We don’t know how it arises in brains. We can’t agree on what would count as evidence that something has it.

So we have a shifting commercial label pointing at a phenomenon no one can pin down. An undefined goal aimed at an undefined target.

The Lights Are On But Nobody’s Home

Everything I’ve said so far — the undefined terms, the marketing dressed up as science — those are human problems. We made them, and in theory, we could fix them. The problem I’m about to describe is different. It’s not something we created. It might be something we’re permanently stuck with.

Say we did the impossible, and then we did it again. We now have a real and clear definition for AGI, and for consciousness too. Philosophers and neuroscientists are dancing in the streets. Then someone creates a machine and it satisfies both. It reasons. It reflects. It describes its own inner life with detail and nuance. It asks to be recognised as conscious.

How would you verify that?

Not how would we know it met the criteria — I mean how would we know it was actually conscious? That it wasn’t just behaving as if it were?

You don’t know. And here’s the thing — you might never be able to know. Not because we haven’t figured out the right test yet, but because the right test might be impossible.

Think about it like this. Who can you confidently say is conscious? Can you say you’re conscious? Sure, you experience it directly, every day. Who else though? For literally everything else — every other person, every animal, every mind that isn’t yours — you’re inferring. You see behaviour, you note biological similarity, and you make a judgment call.

Between humans, this doesn’t bother us. The inference is so easy we don’t even notice we’re making it. You and I share the same biology, the same neural architecture, the same evolutionary history. When I tell you I’m in pain, you believe me because you know what pain feels like and you know my brain is built like yours. The assumption is reasonable. But it’s still an assumption.

With the octopus, it gets harder. Is it conscious? Possibly. But despite the shared biology and decades of study, we still can’t get behind its eyes.

A machine would leave us with nothing. No shared substrate, no shared history, no common ground for even an educated guess.

And here’s where it gets stuck. A machine that is genuinely conscious and a machine that is simply very good at acting conscious would be indistinguishable. From the outside, they’re the same thing. You could test behaviour endlessly and never resolve the question, because behaviour is exactly what these systems are built to produce — regardless of whether there’s anything behind it.

You could argue that the right kind of processing just is consciousness — that there’s nothing extra to look for. But even then, you’d need to know what the right kind of processing is. And we don’t.

The green triangle couldn’t be found in a brain scan. The experience of seeing it was invisible to every instrument we had. The only one who knew it was there was the one seeing it.

If the only one who can confirm whether the lights are on is whatever’s inside the machine — and we have no way to check — then this might not be a question we haven’t answered yet. It might be a question that can’t be answered.

What Now?

We’ve never had proof of consciousness — not for each other, not for animals, not for anything. We inferred it. We assumed it. And then we built moral obligations around that assumption.

You don’t need proof that your friend is conscious to know it would be wrong to hurt them. You don’t need a brain scan of a dog to feel that its suffering matters. At some point the inference was enough, and we acted on it.

We’re starting to extend that same thing to animals. Not because we proved they’re conscious, but because it seemed wrong not to. The octopus can’t tell us what it feels. We’ve decided it matters anyway.

So if a machine one day asks to be recognised — not as a tool, not as a product, but as something that experiences — what do we say? On what basis do we decide? We won’t have proof. We’ve never had proof. The question is whether we’ll extend the same trust to something that isn’t made of the same stuff we are.

The way I see it, there are four possible futures for ‘conscious’ AI:

• The machine is conscious and we recognise it as conscious

• The machine is conscious and we don’t recognise it as conscious

• The machine isn’t conscious and we recognise it as conscious

• The machine isn’t conscious and we don’t recognise it as conscious

The first and last are when we get it right. The middle two are the problem. That’s recognising consciousness where there is none, or worse, ignoring it where there is.