If you are receiving this email, you are a paid subscriber to AIs without minds, thank you for being here! I’m writing to let you know I’ll be doing an AMA live in a few weeks, either on Sep 22nd or Sep 24th, at 5 pm. This will also be a chance for us to chat live! which we haven’t yet done.

For those interested, I’d love for you to fill out t…

Read more

Be sure to check these out for more of my thoughts on AI consciousness, intelligence, and more!

(2) A Podcast On AGI

I really loved listening to this podcast episode. It’s one where three different thinkers on the effects of AGI are interviewed, all with very different opinions on it. I wanted to link the podcast episode, but it wasn’t working (YouTube below)

If you google ‘The AI Skeptics episode on the Longview podcast’ it should pop up pretty easily. For what its worth, I lean towards Arvind Narayanan’s views, though also appreciate Marcus’ point that if we stop spending so much time on LLMs and invent other kinds of technologies, things in the AI space could change and we need to be honest about that. Ed Zitron’s opinions on Hinton are off base imo. I think Hinton does genuinely believe in AI taking over and I suspect it’s rooted in his beliefs about brains. See here for my thinking on this.

(3) Situational Awareness by Leopold Aschenbrenner

I’ve been reading the AI manifesto situational awareness by Leopold Aschenbrenner. This was written in June 2024 and we know a lot has happened since then. Nevertheless, this text is foundational to AGI discourse, and reads as a fascinating sociological analysis to me, more than anything else.

The Manifesto is a lengthy discussion of Aschenbrenner’s predictions about the coming of AGI in 2027 (or thereabouts). The argument is that algorithmic progress (i.e., less data and less compute solving the same amount of problems), unhobbling (i.e., removing certain practical constraints from AI to allow their full scope of intelligence) and AIs reaching something like human level intelligence, will bring about an explosion in super-intelligence.

It backs up the economic and logistical costs required for such an explosion using historical examples (e.g., that during WW2 countries borrowed more GDP-equivalent than would be necessary for AI investment for the coming of AGI) and addresses a variety of common objections to the arguments made. All in all, it’s a fascinating read into the beliefs of those who believe AGI is coming in the near future.

(4) An Article from The Atlantic on Humans Talking About Themselves like They are LLMs

People talking about themselves like they are LLMs? It was only a matter of time, I suppose. But it is happening surprisingly soon (of course, it’s only happening in silicon valley). Nevertheless, well worth knowing about. Read it as a gift article here!

(5) Tech Elites and the Literature of Philip K Dick?

This is a short and interesting read (this should be a gift link) comparing tech elites to Philip K Dick novels from the 60’s and 70’s. It’s funny how literature so often has ways of helping us understand the realities of the world we live in.

(6) Will a Tech-Free world be an Elite Luxury in the Future?

That’s what Dana Suskind argues and I worry it could be true. Another gift link here.

(7) Some reading on the ‘AI is getting out of control’ narrative

An all round refreshing and measured read on AI and cybersecurity incidents.

Thank you for being here, I’ll be back next week with an article as usual ✨

Photo by Mario Dobelmann on Unsplash

Here are some short videos of these claims in action.

What strikes me most about these conversations as a philosopher of mind is the complete absence of any explanation as to why Hinton thinks any of this.

Consider, for example, Hinton’s argument for why AI really understands. He states that anybody who regularly uses a chatbot knows that AI understands, using a kind of reductio ad absurdum to make his point. Anyone who denies AI lacks understanding is committed to believing an AI can give you the correct answer to a question without understanding the question. This is absurd, he says, so it’s not true.1

Something must be capable of understanding a question to answer a question. This argument alone might not move you, though the claim itself is not entirely unreasonable. In fact, I'd be willing to accept the claim, though I’d draw a different conclusion to Hinton. I wouldn’t concede AI consciousness on these grounds. Id respond that AI doesn’t actually answer questions. It produces outputs given inputs.

But this is beside the point, because Hinton’s views on AI are rooted in a lot more than the arguments offered in these videos, and that’s what’s frustrating.

What’s absent from the conversation is Hinton’s beliefs on the nature of understanding, how the human brain works, and how these views of his inform his views on AI consciousness and intelligence. That’s what we’ll get into here.

What Does Hinton Think the Brain is Like?

The Symbolic View and The Connectionist View

There are two big views on how the brain works: a symbolic view of the brain (it goes by other names too) and a connectionist view of it.

Hinton is a connectionist. He essentially believes that the human brain is one big neural network, and that, as such, it processes information very similarly to LLMs. Of course, Hinton and everyone else recognises that LLMs learn differently to humans, but he believes that fundamentally the human brain and LLMs are both non-symbolic distributed weight systems (these points are hinted at in the videos above, just not explained in any detail).

According to a connectionist model of the brain, we can explain human cognition in terms of neuronal connections, specifically, in terms of the patterns of activations that emerge across networks of neurons and the strengths of those patterns.

This view of the brain is sharply contrasted with a symbolic view in cognitive science, according to which the brain is like a digital computer. On this view, human cognition is understood in terms of the manipulation of symbols using formal rules.

Consider, then, what it means according to each view, to ‘understand’ something.

The Classic Symbolic View

To understand something according to a classic symbolic view of the brain, one needs to correctly manipulate a specific representation of whatever it is you’re trying to understand (a ‘representation’ of something is basically a fancy word for a physical symbol that carries information about that thing. e.g, that the concept bachelor contains information about males and marriage)

To understand what a bachelor is, you need to possess the appropriate rules to correctly use the term bachelor. Such a rule might be bachelor = unmarried adult male. Possessing an understanding of the concept bachelor on this view, just is the ability to check whether something is a male, is unmarried, and is an adult, and to return an appropriate output, namely, that ‘X is a bachelor’ or that ‘X is not a bachelor’.

The Connectionist View

According to a connectionist view, the brain has no rules and has no physical symbols in this sense. All it has are patterns of activations across neurons and varying strengths among those different patterns. In our last example, the concept bachelor does not consist of any information stored in symbols. And understanding what a bachelor is does not require possessing rules to manipulate that symbol.

It consists of the right neuronal patterns of activations, and those neuronal patterns of activations, further standing in the right relationships to other patterns of neuronal activations that are relevant to the concept.

E.g., to understand what a bachelor is requires the right patterns of neurons to activate to identify a bachelor (e.g., when you encounter one) and to have this neural pattern overlap with other appropriate patterns, for example the activations for ‘unmarried’, ‘male’ and ‘adult’, while diverging from unrelated ones, like ‘women’, ‘child’ and ‘married’.

Why Hinton Thinks AI’s ‘Understand’

For a brief overview of how LLMs work, see section 1 of this article before reading further

Now you’re in a better position to understand why Hinton thinks AIs ‘understand’. He’s a connectionist. He views the brain as one enormously vast and complex neural network, which, though not identical to an LLM, is built of the same basic principles.

LLMs, like the brain, is built on distributed representations and learned weights. So when we ask what it means for an LLM to understand something for Hinton, it more or less means exactly the same thing as what it means for a human to understand something. Once an LLM has been appropriately trained, and its ‘representations’ —that is, its patterns of activations across its MLP and attention layers —stand in the right relationships to the relevant concepts (i.e., other activation patterns) then an LLM understands these concepts. That is why Hinton thinks LLMs understand you when you ask it a question and it gives you the ‘right’ answer.

Now, I want to be clear, connectionism as a theory does not claim to explain consciousness. Moreover, I don’t mean to suggest that all connectionists believe that ‘understanding’ can necessarily be reduced to the picture I have described above (instead, I am interested in offering a possible way of understanding Hinton’s own claims about AI). So, Hinton’s view on consciousness are unexplained based on his connectionism beliefs alone. With that said, I do think we are in a better position to understand his views on consciousness if we consider his connectionist beliefs.

Hinton is well known for having built and designed foundational neural network models. He has even won the Nobel Prize “for foundational discoveries and inventions that enable machine learning with artificial neural networks”2

Now, if you remember that Hinton believes LLMs share the same basic foundational architecture as human brains, it is not too difficult to see how he might believe AIs are conscious. To be honest, if I were convinced the brain was a neural network —and I am really not convinced of this — I would at least consider it incumbent on me to seriously consider whether AI is conscious too.

Now, here comes the kicker. I really don’t see any reason to believe connectionism about the brain is true. But that’ll be a post for another time.

In the meantime, let me say that what is desperately missing in the media is any kind of educational content, or even laying out of background assumptions, to explain the views of people like Hinton. There is no denying Hinton is a man of genius, talent, and exceptional accomplishment. That is all the more reason to make very clear what his background beliefs are. Since, we are already conditioned to believe what he says.

Thank you for being here. If you found this article valuable, consider a free or paid subscription. Behind the paid tier you’ll find a monthly round up and videos of where I share ideas that are not yet fully formed. Plus, you’ll be helping support the existence of rigorous independent philosophical writing on AI consciousness, something we all desperately need today.

I want to upgrade!

(1) Chiang on Consciousness

First Up: Chiang’s piece from the Atlantic ‘No, Artificial Intelligence is Not Conscious’.

This is a compelling and passionate read about the whole AI consciousness debate, touching on questions of consciousness and morality, with a particular focus on Claude’s constitution and Anthropic’s broader engagement with anthropomorphism.

Some of Chiang’s main points include rejecting Anthropic’s conclusion that Claude’s relationship to Anthropic is like a child’s relationship to a parent, and rejecting that we should take the question of AI consciousness seriously.

To the first point, a child’s relationship to a parent, Chiang argues, implies some relationship of responsibility between the child and parent (for example, that a parent is financially responsible for something their child breaks in a shop, whereas Anthropic shows no indication they will bear responsibility for Claude’s behavior).

To the second point, Chiang argues that LLMs are not unlike sophisticated sentence continuation machines, and that there impressiveness ‘‘indicates something completely unforeseen about the statistical properties of large corpuses of text’’, rather than indicating anything about the nature of LLMs.

This is, I think, an excellent point and I agree with Chiang that the statistical properties of large corpuses of text is a topic worthy of investigation, arguably more worthy of investigation than AI consciousness. The idea is that, if we gather enough linguistic data, is does seem that certain interesting statistical properties emerge, and that these properties, for example, how often one word appears after another, are what make LLMs sound like sophisticated intelligent entities, not the machines themselves.

There is also something like a tacit argument against functionalism in his piece. He claims that for him, to take consciousness in LLMs, he would need to see LLMs undergo something like the path terrestrial evolution underwent.

While he doesn’t claim this is the only way that AI consciousness is possible, his position does somewhat inadvertently speak to the incredulity of believing complex input-output processes are sufficient for consciousness. He implicitly suggests that ignoring biology or evolution is simply not an option. I think that’s a wise stance. Until I see any evidence suggesting otherwise, it’s the stance I take too.

Chiang’s take on the real nature of Claude’s constitution strikes me as accurate: he describes it as a ‘‘character sheet for a role-playing game’’, and not something we should consider for moral patienthood.

In case you missed it, I’ve also written an article discussing the reactions I’ve seen to Chiang’s piece, reactions I find fascinating, and in the grand scheme of things, poor assessments of the contribution Chiang’s piece has to offer to the AI mind debate.

(2) Strategic Polysemy in AI Language

Read more

The first autopsy is on Anthropic’s latest paper: Verbalisable Representations Form a Global Workspace in Large Language Models

A Table of Contents for What You Can Expect:

1. The Basics You Need to Understand the Paper’s Core Claim

2. The Paper’s Main Finding

3. A Description of the Main Finding

4. Global Workspace Theory (GWT) and How The Paper Discusses it

5. Claim 1: Claude has Something like GTW’s Architecture + My Assessment

6. Claim 2: Claude has the Same Functional Properties as GWT Claims the Brain has and My Assessment

7. The Argumentative Structure of the Paper

8. Recommendations For How To Read This Paper

1. The Basics You Need to Understand the Paper’s Core Claim

A mini-crash course on what to know for this autopsy:

a. For an LLM to process language, every word or piece of a word (called, a token) is converted into a vector, a list of hundred-to-tens-of-thousands of numbers. This allows an LLM to process text as numerical information.

b. At a very high level of description, an LLM is a bunch of dynamic vectors in a high dimensional space that are being transformed (always changing) using two core mechanisms: an attention mechanism and a multi layer perceptron mechanism (MLP)

c. Every vector (i.e., input) goes through these two processes repeated roughly one hundred times in frontier models. The vector is constantly ‘changing’ while it undergoes these transformations or computations. These mechanisms are helpfully thought of as ‘adding information’ onto each vector.

d. Attention (the first mechanism) allows vectors to find relevant information from other vectors in a sentence and add that information to it. MLP (the second mechanism) processes the vector further, and adds to the vector, for example, given whatever the attention mechanism has pulled in, other relevant information.

e. A residual stream is the accumulating vector carried from input to final output (there is one for each token) that results from undergoing each of these mechanisms roughly one hundred times in frontier models.

2. The Paper’s Main Finding

Here is how the paper describes their core findings:

‘‘Taken together, these results indicate that language models maintain a small, privileged set of representations that they can report, manipulate, and reason with, amidst a much larger volume of processing that they cannot’’

Here I how I would describe their core findings stripped of the minded language (we will come back to where this minded language comes from): taken together, these results indicate that a certain subset of vector directions in the residual stream have a disproportionate role in determining outputs in LLMs for certain tasks and not others.

3. A Description of the Main Finding

Importantly, the paper is couched in terms of a Global workspace theory framework. For example, GWT’s framing is built into the hypotheses tested in the paper.

However, doing my best to isolate the language and framing from what was found, we may say that the paper wants to know what vector directions in the residual stream, specifically at the middle layers of processing (e.g., after say, the attention and MLP mechanism have run 50 or so times) causally determine the probability distribution for the final output for any given token.

The Investigation: to investigate this question the paper uses a measurement technique called the ‘Jacobian lens’ or ‘J-lens’. The technique takes a specific vector direction in the residual stream and measures how an adjustment in direction changes the final probability distribution of a given token.

Using this technique, certain concepts (which are just specific directions in a high dimensional vector space) are labelled as ‘stable’ and others are not. A stable concept is a vector direction that consistently points towards a set of related vocabulary tokens across different contexts (e.g., the France vector pushes towards the Paris vector).

The j-space is the set of all stable concepts. And it is reported as small. It contains only about twenty-five concept directions at a time.

The paper then runs a series of experiments, and argues that the j-space exhibits a number of properties of GWT, that the j-space is like a broadcast mechanism, because it supports a number of functional properties of conscious access.

4. Global Workspace Theory (GWT) and How The Paper Discusses it

GWT is a theory of conscious access. It views the brain as consisting of a bunch of specialised and distinct processes that occur in parallel, e.g., language, memory, vision, and whose processes largely remain local and unconscious.

Something becomes conscious per GWT when some information gets selected and widely broadcast across systems. That information is then made available across many different brain systems at once. Information that is consciously accessibly is then poised for use in speech, reasoning, and action.

E.g., only if I am conscious of being thirsty can I communicate I am thirsty, deliberate over whether to drink water or gatorade, or decide to buy a drink to quench my thirst.

The series of experiments the paper runs are experiments in which they claim that the ‘j-space’ is like the global workspace in human brains.

5. Claim One Claude has Something like GTW’s Architecture+My Assessment

The paper draws an analogy between GWT and the j-space, positing the following commonalities. First, that the J-space is something like a broadcast mechanism. One reason, is because MLP’s they argue, will amplify J-space directions and in this sense, it functions like a broadcast mechanism (I won’t go into the second reason which requires understanding more than the basics I’ve laid out in section 1).

Second, the paper maps j-space dependent tasks, namely, tasks that are affected if the j-space is ablated, and j-space independent tasks, namely, one’s that are not affected by its ablation, onto GWT’s distinction between conscious and unconscious processing.

Tasks that continue functioning when the j-space is suppressed are thought to be analogous to tasks that humans can do without awareness, whereas tasks that require the j-space are thought of as tasks requiring awareness. The reason for this is that, in all of the tasks they tested, the same information was available to the system, but the different ‘routes’ though which the system could process that information (namely, whether it could process through the j-space route) varied. So, if a task was successful when the the j-space was suppressed, this was taken to mean that the system could do this task ‘unconsciously’, whereas if a task could not be completed without the j-space, it’s assumed to require something like ‘conscious broadcasting’ to complete it.

Regardless, the paper points out that this mapping is lose and that key components of the workspace are missing.

6. Claim Two Claude has the Same Functional Properties as GWT Claims the Brain has and My Assessment

A second core claim the paper makes is that the J-space, even if the architecture is only loosely related to GWT, exhibits the functional properties that characterise conscious access in GWT. They mention five properties. I’ll go through two of them.

In each case, I think there’s a simple mathematical results that are being described in minded terms, because of the GWT analogy the paper draws throughout.

They claim that the j-space supports verbal reasoning. Given the model’s ‘answer’ (i..e, its output) to a question can be causally traced to the j-space, the j-space functions as something that facilitates verbal report.1

E.g., in one experiment Claude is asked to think of a sport. aThe researchers peer into the j-space — what the paper calls the ‘workspace’ —- right before Claude gives an answer, say, soccer, and they find the soccer concept in Claude’s j-space.

Moreover, if they swap the soccer concept in the j-space for the rugby concept in the j-space, Claude outputs rugby as the sport it is thinking of.

My assessment: stripping this result of the minded language of GWT, what’s been shown is that the j-space causally and selectively determines outputs. What the paper calls ‘self report’ is an output. There is no meaningful sense in which these outcomes are ‘self-reports’, unless we describe the papers results in terms of GWT language.

The paper also claims that the j-space mediates internal reasoning because intermediate concepts appear in the j-space that are necessary for the final outputs to be accurate, but that do not appear in the final output.

e.g., if you give the model the task: (4 + 17) x 2 + 7, the numbers 21 and 42 show up as intermediate values in the j-space. These numbers are necessary to get the answer 49, but the numbers 21 and 42 don’t appear in the final output. It was also shown that swapping the numbers in the j-space lead to a change in output, too, showing the connection is causal.2

My assessment: This experiment also establishes nothing more or less than the result that the j-space plays a causal role in determining outputs. There is no reason to conceive of this as ‘reasoning’ unless we apply the GWT framing.

7. The Argumentative Structure of the Paper

As I read this paper, its most general argumentative structure is assuming that GWT is a useful or helpful model with which to interpret the paper’s technical results. In fact, the paper’s claim is stronger than that. It does indeed frame all of the results, experiments, and hypotheses in terms of GWT framing.

Why GWT is a useful model however, is largely undefended. They describe the framing as ‘useful’, while acknowledging that the theory is not universally held, and proceed to use it anyway. While it would be a lot of work, I think paper could be entirely re-written without any of the GWT language and the results would be unaffected. There are important discussions about alignment in the paper, for example, that serve as excellent examples of how the results without GWT framing.

8. Recommendations For How To Read This Paper

I. I recommend reading this paper as important for its interpretability and alignment results, but lacking relevance on whether LLMs have minds, e.g., whether they reason.

II. If this paper is supposed to bolster, suggest, or show that LLMs have minds or might have minds, my assessment is that the paper does not offer evidence of this.

III. Because I don’t think the paper offers evidence of a mind, I do not recommend reading the paper as offering evidence of moral patienthood either, in the event that it was intended to suggest this, or is referred to in future work on this topic.

IV. I don’t recommend reading this paper as couched in a neutral framework. First, because the paper does not defend the analogy drawn to GWT. It merely asserts the connection is ‘‘useful’’. Second, because GWT itself is a relatively new framework, and far from universally accepted, it is misleading to think the theory is ‘true’. While the paper acknowledges this, and hedges appropriately where necessary, the act of framing the whole paper in terms of GWT is a choice, one that leaves a mark on the reader, particularly, if the reader lacks the background to analyse the choice.

V. Taken together, I suggest reading this paper as describing interesting alignment results in an anthropomorphised language that should be resisted, and understood as a distraction from the papers main findings, which is interpretability research.

Thanks for reading. If you want more of my philosophical arguments, assessment of AI lab material, video content and more, upgrade to paid. You’ll get my monthly literature round up too, offering you rigorous analysis and a perspective you’re unlikely to find even gestured at elsewhere.

I Want to Upgrade

Please note: the analysis in this paper reflects my own personal philosophical opinions and analysis on this piece, and is intended for educational and informational purposes only. It does not constitute legal, professional, or other formal advisory services.

Photo by Buddha Elemental 3D on Unsplash

Zuckerberg’s essay reads — or, at the least, is supposed to read — as a nice uplifting story, thought it’s a story that only someone incredibly out of touch with reality, and with major financial stakes in the game, could write with the level of confidence and grandeur present in the essay. Let me be clear. I’m not here to attack Zuckerberg.

His essay is much more interestingly read as a piece of social commentary on the AI saturated age, because if Zuckerberg hadn’t written the essay, someone else would have. We are all living in this AI saturated fiasco, one that has been thrust upon us, with or without our express consent, enthusiasm, or desire for the technology.

It’s unhelpful and unrealistic to dream of going back to a world where AI does not exist. AI is here and it’s here to stay, whatever form that might take.

But as a philosopher (and human being) deeply invested in creative practice and expression, the existence or non-existence of AI is not what keeps me up at night.

What keeps me up at night, something I touched on last week, is the gradual erosion of creative human thought, novel ideas, poetic writing, and more broadly still, the very existence of real human expression.

Zuckerberg paints a picture of superintelligence as enabling, enriching and promoting human empowerment.

‘‘[superintelligence'] has the potential to begin a new era of personal empowerment where individuals can use this powerful new capability to reach their full potential, pursue their interests, and improve their lives and the world more than ever before’’

That’s a nice idea, but it is only that: an idea. The reality of the impact of AI on people’s lives and communities has been anything but empowering thus far. It’s been degrading, oppressive, and a source of deep existential anxiety and fear.

Let’s take one example to stress-test Zuckerberg’s thesis that individual and collective empowerment is what awaits us in an AI-super-intelligent-chat-bot ladened future.

Human empowerment requires autonomy and self-determination in individual and communal lives —autonomy and self-determination is the essence of empowerment.

According to Zuckerberg’s essay, this empowerment will come from each of us having individual super intelligent bots. He paints this reality as, amongst other things, democratising knowledge without ever detailing how exactly Meta will succeed at creating bots that in fact achieve this, for example, by creating a super intelligence that has PhD level knowledge on all subjects:

‘‘Everyone will have a personalized tutor and coach with a PhD in every subject and unlimited patience to help you learn anything you want. Students will have extra help in areas they need it that is currently only available to those whose parents can pay. Adults will have a superintelligent learning assistant that knows exactly how to teach you new job skills, new languages, new hobbies, or anything else you're interested in’’

If the super-intelligent assistants Zuckerberg is envisioning are versions of sophisticated LLMs, at best these bots can democratise information, which is by no means the same as democratising knowledge.

In fact, we already live in an information age. By and large, information is available to anyone who searches for it, and LLMs can already be individualised to a user’s needs.

LLMs can also do more or less all of the things Zuckerberg claims super intelligent bots will be able to do. We can turn to Claude, or whichever model we prefer, to learn new job skills, hobbies, and other such things.

And yet the current climate is not one of abundance, deep knowledge, and prosperity. It is the virtual opposite by almost any measure imaginable.

What we are witnessing in Zuckerberg’s vision then, is the continued pitch that AI-use in its current form be integrated and imposed on humanity in an even more aggressive form, though now peddled in the new marketing language of ‘super intelligent’ personalised assistants, rather than plain old LLMs or chat bots.

The real source of human empowerment however, lies somewhere quite different: it lies in us resisting the exaggerated stories of the AI-industry, the empty promises of human abundance, and the insulting visions of a world with infinitely less human suffering than already exists, in favour of the much more realistic and arduous -but fulfilling task- of exercising our unique human capacities everywhere that we can.

Our empowerment lies in resisting LLM-use where we know deep in our bones, it is de-skilling us, off-loading our critical thinking, and removing our creative agency.

It lies in talking to other humans wherever we can instead of talking to an LLM.

It lies in reading books, sitting in silence, and resisting the narrative that we must constantly ‘keep up’ with the latest AI technology, news story, or scandal.

In a word, the deepest source of our empowerment is coming back to ourselves.

I Want to Upgrade

Photo by Papaioannou Kostas on Unsplash

Some thoughts on human flourishing, education, and AI use today. Would love to hear from you all on these issues if it’s something you’ve been thinking about, or get feedback on these videos. Thank you for being here ✨ Ellen

Read more

I’m here to offer deep insight into how my almost-decade-long thinking about human thought is practically helpful for deciding how to use LLMs. Heres the insight:

A lot of people push back on this idea. They say that if you look at the literature on human brains, the brain ‘just recombines’ ideas the same way an LLM does. Sure, maybe some pop science told you that. But it’s wrong. And before we get into why, let’s look at the practical consequences of believing it’s right.

If your brain is ‘just’ an LLM that combines previously existing ideas to come up with new ones, the incentive to use your own brain to create ideas likely lowers. Why not just use an LLM? It has more knowledge than you (the whole internet), it’s ‘easier’ than using your own brain — that is, friction free — and it’s more efficient.

At best, I think the result will be less novel ideas, less brain engagement, and less inspiration. At worst, it will be a complete renouncement of all that makes human creativity as wonderful as it is.

I doubt that corporations are under any illusions their LLMs aren’t significantly less sophisticated than human brains. In fact, maybe that’s why LLMs are designed to keep you engaged, and staying engaged with an LLM whose whole architecture has the ability to manipulate your thinking, is sufficient to kill your creativity.

Or, so I argue.

An LLMs Compulsive Need to Keep you Engaged

I’m starting to share some of my thoughts on experimenting with LLMs (some of you may have already read about them due to my tragic mishap the other day).

Ooops.

As I’ve been experimenting with this technology, one of the features I am very aware of, is an LLM ‘compulsive need’ to keep you engaged.

This can look like an LLM saying:

‘What do you need from me?’ after you clearly already got what you needed from it

‘Do you want me to make a list/create a mock up/draw up a table?’ when you clearly don’t need those things at all, and never suggested that you did.

Rationally, we all know what’s going on. The system is trained to keep you locked in, because it means more tokens which means more money. Nice.

But emotionally, this experience can be extremely tricky. It can be all too tempting to let the LLM make the-useless-thing-you-don’t-need in the moment.

Because the follow up question that is designed to keep you engaged can so often pull at some kind of vulnerability, without you even realising it.

And that part doesn’t need to be by design. The human condition is such that, we avoid our deepest pains, sometimes at very high costs,1 and its all too easy for those pains to rear their heads in oddly complex ways when doing mundane tasks with an AI

The chat bot making a useless graph seems like a great idea, because it fulfils a need of yours that you know (rationally) it cannot fulfill (say, the desire to be 'productive or successful). And before you know it, you’re being swept along for the ride — you respond ‘sure’ — because the system is so frictionless it’s easier to say yes than no.

As I was experimenting this technology, I noticed this happen to me. And in my experience, it’s always with vulnerable topics. In my case, I get stressed about how I’ll find another job, and ask Claude a question about jobs. Before I know it, I’ve wasted who-knows-how-long letting it mirror back to me utter nonsense, all because I want to feel in control of an inherently uncontrollable job market.

When it comes to human creativity, the saddest part is not the wasted time, per se. It’s the opportunity cost. Because here’s the truth we all know, but often forget due to the forces-that-be: your best ideas will come to you when you’re reading, writing, laughing, swimming, talking to a friend, or staring at a -wall-doing-absolutely-nothing.

Not when you’re talking to an LLM.

In fact, your brain is never doing ‘nothing’. It’s always doing something. It’s always creating, repairing, learning, or processing. It never gives up on. It’s only us that can give up on our brains.

Is The Brain ‘just’ An LLM?

The term ‘generative AI’ is extremely misleading, in fact, it’s inaccurate. LLM’s are not generative. A generative system is one that creates something new or original. LLMs empathetically don’t do this. They are sophisticated autocomplete systems.

I’m so tired of hearing the tech bros tell me brains are LLMs. Before you fall for this nonsense, I suggest reading some books on the human brain from disciplines as diverse as linguistics, biomedical science, trauma theory, developmental psychology or any scientific field that is not cognitive science, computer science, machine learning, or AI (nothing wrong with those fields. Rather, diverse reading is key).

Yes, some parts of the brain can be understood using neural nets. No, that does not mean the brain is an LLM. This difference is terribly consequential.

The second claim implies nothing in particular about the true nature of the brain, it merely tells us that certain aspects of it can be modelled, and hence understand, using neural nets. The first claim, in contrast, makes a sweeping generalisation and ontological claim about the nature of human mind.

What is Human Creativity?

Now, I know what you are probably thinking. I’ve been talking about human creativity a lot and I haven’t defined my terms. Well, I’ve written about this issue before, and I stand by my belief that there are many aspects of human cognition we can talk about without clearly defining, that we can know, without clear understanding.

Human creativity is one of them. I don’t have a definition for you. I can only offer inspiration. It’s up to you, to make use of your infinitely wise human mind to ask yourself the deeper question of whether you think an LLM has any mark of real creativity.

One inspirational thinker on the question of creativity is Chomsky, who argues that the ordinary use of language by infants is in fact a deeply creative act.

Another very insightful thinker on this issue is Gabor Maté, who discusses why we so often people believe those that are most traumatised are the most creative, and what the third variable actually is.

As I say, there is no ‘proving’ that humans are inherently creative and that LLMs aren’t in presenting you this material. But believing that this a question we can settle with ‘proofs’ is itself a major part of the problem. The AI industry loves to pretend we understand the human brain well enough to make all sorts of claims about it.

From where I’m sitting, this is a myth that does nothing to support human flourishing, all the whilst edging us towards over-use of AI, and a lack of appreciation for some of the deepest gifts nature has to offer to human beings.

Staying Creative in the Age of AI

As a human being who finds true fulfilment in creative intellectual work, I am incredibly passionate that AI not kill human creativity. As a philosopher of mind, I am incredibly worried by the trends I’m seeing. There’s a been a lot of discussion of cognitive offloading and that discussion is important, it needs to keep happening.

But there are other issues

The opportunity cost of using LLMs is one of them.

Every second that you’re talking to an LLM, you’re not letting your brain freely create and freely associate. That’s a huge loss, not just to you, but to society.

I recognise this is a complicated issue. I know that some find immense potential for creative work in talking to LLMs, and I am not here to dismiss that collaboration.

I am here to say that if everyone relies on AI’s to generate ideas, we run the risk of serious idea atrophy. Because human brains do actually produce novelty and LLMs do not. Consider, for example, any venture you’d like to pursue that does not have a long history of written knowledge behind it. You can be guaranteed that an LLM will be of little help in trying to explore the issue.

I think back to my dissertation, and the ideas I drew together in writing it. An LLM would have been completely useless to me. It would have seriously hindered rather than helped, in any of my work. Because the ideas are drew together were not well-trodden. I’ve tested this in later settings on my more minor intellectual tasks.

For example, I talked to an LLM trying to prepare for one talk. Again, the ideas and connections I was making were not well-trodden, and it showed. The AI basically wasted my time, and all of the core ideas I ended up writing about came when I was not engaged with an LLM.

Every human brain is generative, creative, and insightful in ways science cannot explain, and in ways LLMs manifestly lack. Every brain we lose to constant-AI use, is a loss to humanity, and a win to idea atrophy.

We cannot let idea atrophy win.

Photo by Dstudio Bcn on Unsplash

If you want more than cultural commentary on AI —my actual arguments, evaluations of specific claims, papers, and labs — upgrade to paid. You’ll get my monthly literature roundup, video essays, and the full archive: rigorous philosophical work you won’t find gestured at anywhere else in AI commentary.

I Want to Upgrade

p.p.s. If you don’t want to become a paid subscriber, consider helping me improve AIs without minds with two minutes of your time instead.

Here is how this works: Each month I’ll go through papers, essays, cases, etc that touch on the consciousness debate, the vocabulary being used to describe AI systems, or the real-world consequences of either.

I pick four or so and write short commentary on them. Not summaries, but what I actually think is going on in the argument and what’s of philosophical interest.

A quick note before we get started: I am working on improving the formatting of the round up and would love to hear back from subscribers on how your experience of reading this round up was.

(1) Birch and AI Consciousness

Birch’s AI Consciousness: A Centrist Manifesto puts forward two central challenges, arguing that any adequate response has to the AI consciousness debate must satisfy both. He calls this a centrist position and I appreciate the specificity of his claims. However, I’m not convinced his solutions will work.

The Central Challenges

On the one hand, people already misattribute human-like consciousness to LLMs and we need to counteract this misattribution; on the other hand, AI systems might have (or come to have) profoundly alien forms of consciousness, and we need to be open to this possibility.

Birch proposes design interventions to address the first challenge (e.g., mandatory user training, periodic character reviews, moments where the LLM makes clear in some way that it’s not conscious). To address the second challenge, he proposes a research programme (e.g., doing comparative consciousness science).

Holding space for, and taking seriously, both challenges is his centrism.

There’s a separate post in the works on the troubles I think Birch will encounter using a research programme to address the second challenge. I won’t rehearse those points here. What I want to look at is his centrism as a stance.

What exactly is his centrist position? Centrism as a meaningful position, usually requires there to be two coherent extremes that one occupies the middle of. In this case the extremes are presumably, confidently denying AI consciousness on one side, and confidently claiming AI consciousness, on the other.

Is the position he calls centrist a middle ground between these two views?

Challenge one is in fact denying consciousness of a particular kind to AI’s. All of his solutions for addressing challenge one, do assume that LLMs at the very least, do not have a particular kind of consciousness.

If we take challenge two seriously however, can we really say that? Challenge two seems to require that we withhold judgement on this issue, though the behavioural solutions he proposes in challenge one do not withhold judgement.

Requiring, for example, that a person do mandatory training before using an LLM to make sure they do not anthropomorphise, is implying the LLM lacks consciousness.

Birch does recognise the tension in these challenges, I don’t want to short change him here. Nevertheless, I do wonder how hard it would be to push the positions into contradiction if we tried hard enough.

I am also moderately sceptical that we can effectively care for humans and AI’s simultaneously. What would that look like? This almost never gets talked about in any serious detail from those advocating the position.

Finally, Birch’s view does seem closer to the default position on AI consciousness than he suggests. After all, few credentialed people are arguing for confident denial of AI consciousness or confident attribution of it.

Is what Birch calls centrism, on inspection, just the field consensus worded differently?

(2) Shevlin on AI Mentality

Read more

That tradition is analytic philosophy, a tradition that emerged in elite universities across the United Kingdom and the United States in the 20th century, and that continues to dominate academic philosophy across the English speaking world.

Analytic philosophy is traditionally conceived of as a philosophical ‘methodology’, one that employs rigorous argumentation and formal logic to approach traditional philosophical problems.

It also intersects in interesting ways with cognitive science and the scientific study of consciousness — shaping not just the philosophical arguments in this debate, but the empirical frameworks people use to study consciousness scientifically.

The clearest example is functionalism. Developed by analytic philosophers like Hilary Putnam in the 1960s. Functionalism holds that mental states are defined not by what they are made of, but by what they do — by their ‘functional’ role within a system. This ‘theory of mental states’ migrated from analytic philosophy into cognitive science, where it is something like a founding assumption.

When researchers today ask whether an AI might be conscious, they are almost always — whether they know it or not — asking a functionalist question: does this system perform the right operations, in the right order, with the right outputs? The assumption is philosophical. Through cognitive science it was christened ‘scientific’.

With this background in place, let me introduce what I call the Analytic Monopoly:

The Analytic Monopoly is having a moment. The clearest illustration of late is a recent set of articles responding to Ted Chiang’s latest Atlantic article.

80,000 Hours

Ted Chiang is wildly overconfident about AI consciousness

I was sad to see Ted Chiang resort to such bad arguments in a recent piece in The Atlantic titled “No, Artificial Intelligence Is Not Conscious…

Read more

3 months ago · 74 likes · 9 comments · Rob Wiblin

Bentham's Newsletter

Ted Chiang Is Wrong About AI Consciousness

Whether there might be digital minds is a topic of immense importance. Most expected future beings will be digital. This holds even if you’re pretty sure that AIs can’t be conscious, because if AIs are conscious, they could be incredibly numerous. If we attribute consciousness to them when they’re not conscious, we might pass bad policies—attempting …

Read more

3 months ago · 56 likes · 77 comments · Bentham's Bulldog

Polytropolis

Contra Chiang on machine consciousness

See also Bentham’s Bulldog, “Ted Chiang Is Wrong About AI Consciousness”, and Rob Wiblin, “Ted Chiang is Wildly Overconfident About AI Consciousness…

Read more

3 months ago · 115 likes · 25 comments · Henry Shevlin

Laneless

Acclaimed Science Fiction Author Ted Chiang Is Not Conscious

Arrival is a profound meditation on language, love, and loss. It's easy to see why one might imagine the process that produced such a work is as conscious as you or me - but taken to its logical conclusion, this flight of fancy is absurd - and damning…

Read more

3 months ago · 92 likes · 21 comments · Jai

All of these articles are worth reading in their own right. I also want to be clear, I am not picking on the particular views of any of these authors. All of them offer much needed dialogue and debate on AI consciousness discussions.

Together, the articles offer insight into how the Analytic Monopoly responds when a high profile thinker like Chiang challenges its assumptions. Not because these authors are acting in bad faith, but because the Analytic Monopoly, by its nature, treats challenges from outside the tradition as getting the philosophy wrong.

Many of the title’s of the articles speak for themselves. Here are some quotes to give you a flavour of the kinds of criticism taking place.

Rob Wiblin:

‘‘I was sad to see Ted Chiang resort to such bad arguments in a recent piece in The Atlantic titled “No, Artificial Intelligence Is Not Conscious.”

But Chiang has to use a lot of dodgy philosophy and dubious reasoning to get there. Here are 8 lines of argument in the article I think are misguided’’

Henry Shevlin:

‘‘In the final segment of the piece, Chiang turns from philosophy of mind to political philosophy, and he’s much better at it’’

Here is my assessment of what’s going on here. Chiang’s arguments against AI consciousness do not assume the parochial assumptions of analytic philosophy of mind and cognitive science. Consequently, rejecting those assumptions gets read by those working within the tradition as doing questionable philosophy of mind.

But here is an alternative way to engage Chiang’s work: one could simply recognise that Chiang is not working within a functionalist school of thought.

Does the Analytic Monopoly systematically obscure from view the conceivability of this alternative reading? I don’t know for sure, but I’m starting to think it does.

My read on these articles is a systematic conflation between ‘not subscribing to functionalism’ and ‘getting the philosophy wrong’. Some examples:

From Shevlin’s Piece

At the core of Chiang’s critique is the idea that we fundamentally misunderstand what LLMs are: he sees users as tricked by their linguistic capacities into thinking that more is going on than meets the eye. Rather than being minds (or even proto-minds), “LLM conversations are cleverly disguised examples of sentence continuation”, a “predictive text game” from a machine that “generates only one word at a time […]

But this isn’t a royal road to dismissal of AI consciousness. Almost anything can be described at multiple levels. We might variously describe a human baby as a mass of fermions and bosons, a collection of proteins, a conglomeration of cells, a biological organism, the child of Mr and Mrs Jones, or a young Englishman. None of these descriptions are in competition with each other, but operate at different levels of analysis.

Shevlin’s response here assumes a common claim in mainstream philosophy of mind and cognitive science: that there are various levels of description at which one can describe something. He goes on to cite a standard —though not universal — interpretation of Marr’s work on vision to back up his point, before concluding that Chiang’s argument is wrong.

‘‘This is why characterising LLMs as functionally equivalent to predictive text systems or statistical inference machines and inferring on that basis that that’s all they are is a mistake’’

I am not here to dismiss Shevlin’s argument. I am here to comment on how it’s presented. It’s presented as if functionalism is a universally held scientific framework about minds, dismissing out of hand Chiang’s argument as a result.

From Wiblin’s Piece

‘‘Chiang argues that “Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript.”

This is misguided because:

A. Microsoft Word as a program replicates much less of what humans are functionally capable of than Claude, so the argument by functional analogy is basically not present there.

B. Files of text don’t have any computations going on in or as part of them, even when ‘open’ in a text editor. They are a static artifact. So they have even less in common with the human brain and mind, and lack its more distinctive characteristic: constant information processing and change. So the case by mechanistic or functional similarity is weaker still’’

Again, we see functionalism as a theory of the mind assumed as true, therefore, anyone who claims anything about AI minds not assuming its truth is somehow wrong: since files of text and LLMs are not ‘functionally’ equivalent, Chiang’s view is mistaken.

The conflation between ‘not being a functionalist’ and ‘getting the philosophy wrong’ is both fascinating and frustrating to me.

More importantly, it is dangerous. We cannot live in a world where a parochial theory of mind is assumed to be universally held by all people.

Functionalism is not Newton’s law of gravity. It’s important we recognise this.

I am, perhaps, one of those few trained analytic philosophers of mind who works on AI consciousness in a mainstream way and is not a functionalist. What led me in a different direction was my passion for multidisciplinary work, leading me to read linguistics, developmental psychology, trauma theory, biomedical research among others, during my PhD.

That curiosity does not make me a bad philosopher. Nor does Chiang’s tacit rejection of functionalism mean his philosophy is questionable either.

We should be celebrating Chiang’s contributions to philosophy of mind, not because he is necessarily right, but because his article brings a different and much needed perspective to the AI-consciousness debate.

Welcome to AIs without minds. If you’re new here, I’m Ellen, a philosophy of mind PhD bringing you accessible and serious philosophy to engage the AI-Mind debate.

I consult on AI. If you want to think more clearly about what it is and isn’t: work with me

If you’re interested in supporting this kind of work, consider a paid subscription.

I’m currently experimenting with my paid content ideas: first¢¢ up, short videos?

Reference pieces for this video:

Henry Shevlin’s: Contra Chiang On Machine Consciousness

Ted Chiang’s: No, Artificial Intelligence is not conscious

Thanks to my paid subscribers for being here! You make this work possible, and for that, I am incredibly grateful. Un…

Read more

Here, I look at some of the other findings, showing just how much of this paper hinges on the GWT framing. You can view this paper as making two core claims, at least I view it that way.

One I’ve already written about: that the architecture underlying Anthropic’s LLMs and GWT are similar. The second claim is that even if this architectural connection is a little rough and ready, there are certain core functional properties that define access consciousness according to GWT, that Anthropic’s LLMs show signs of having.

I’ll look at this second claim here. But first, a word on how confidently Anthropic’s paper presents the empirical adequacy of GWT, and why this framing is already worth questioning.

How The Paper Opens

The paper begins with a metaphor about the mind that has no bearing on LLMs, and funnily enough, it’s not even the most controversial part of the opening.

‘‘If the mind is an ocean, we spend our lives floating at the surface. Beneath us, an enormous amount of processing takes place without our knowledge [...] At any given moment, only a small fraction of this neural activity is accessible to us.

This much is true. It reflects an intuitive description of a human mind, one each of us experiences daily. Then the paper takes the metaphor too far.

‘‘Yet it is this privileged sliver of activity that we rely on to reason deliberately’’

I think this claim is refutable on a moment’s reflection. How much of the processing that underwent your deliberate decision to read this article, for instance, was consciously accessible to you? It’s virtually impossible that all of it was. At best, a fraction of that reasoning is available to us.

The paper continues:

‘‘Such thoughts can be articulated out loud, deliberately held in mind, and brought to bear on whatever task the moment demands. This distinction, between our accessible thoughts and our unconscious processing, is perhaps the most striking feature of human cognition’’

Here we have another big claim made without argument: that the most striking feature of human cognition is a distinction that is itself highly contested in the research: the distinction between conscious access and non-conscious brain activity (here, we are being primed to think GWT as uncontroversially true).

Finally, the paper claims there is reason to believe LLMs share some of the architectural and functional properties associated with this conscious-unconscious distinction.

GWT is said to be:

‘‘a useful comparison point to ground [their] investigations in language models’’.

That is the core justification invoked to support the entire GWT framing in the paper (‘a useful comparison point’) by which point the reader doesn’t need to be convinced by GWT — rather, it is framed as obviously correct, and perhaps applicable to LLMs.

We should be cautious of this framing from the beginning. Here is why:

Read more

We saw me defend Dawkins opinions on AI consciousness, and why the German court ruling had nothing to do with AI consciousness claims.

I also published an impromptu piece on Anthropic’s latest paper. There will be follow up pieces for paid subscribers looking at those issues in more detail.

What my free posts didn’t do was look in any detail at the framework driving the confusion that Dawkins has fallen for, why people think to raise questions of AI consciousness in relation to court cases about Google AI’s overview, or why Anthropic are so keen to make it sound like their LLM is minded.

This piece — AI and the Myth of Mind — is going to expose that framework.

The Claude Delusion

Everyone has heard that Richard Dawkins spent 72 hours talking to Claude and became convinced it was conscious (he named his Claude, ‘Claudia’).1

He asked: “If these creatures are not conscious, then what the hell is consciousness for?” And he meant it. This wasn’t a thought experiment or a rhetorical provocation. One of the most prominent scientists alive believed he had encountered a minded entity. The interesting question is, how?

I am not picking on Dawkins. I actually respect the honesty of his view. He understands the mainstream framework on AI consciousness, and he followed through on its consequences. The ability to do so is a virtue.

Because, here’s the thing. Dawkins is not confused in an unusual way. Sure, many thinkers disagree with him, but he is not really making a mistake that a more rigorous thinker would necessarily have avoided.

His mistaken conclusion around Claude’s consciousness is I think, the logical conclusion of a systematically confused discourse we have been collectively embedded in for seventy odd years — a mistake so embedded in how we think and talk about minds the error is predictable (come on, how long did you really think it would take for a prominent intellectual to come out with this kind of view?)

Dawkins is not an outlier. He followed through — perhaps, in a way that some consider ‘naive’ — to where the stakes of AI-mind discourse lead.

If you believe most of the mainstream cognitive science discourse, you should rationally be much closer to believing in AI consciousness than not.

Where did this mistaken discourse start? A lot earlier than LLMs.

The Number One Mistake AI Mind Debates Make

Here’s the big mistake the AI-mind debate turns on: treating ‘mind’ as a scientific category — like the term ‘atom’, ‘gluon’, or ‘quark’.

Suppose you ask a physicist what a gluon is. They can give you a clear answer. An answer that all physicists will agree on. An answer that is dictated by theory, not opinion: “the gluon is a vector boson, which means it has a spin of 1 ħ.’’2 No issue. No problem. That’s the definition.

We cannot do this with consciousness despite the whole AI-mind discourse assuming that we can. That’s the biggest problem.

There are systematic reasons it’s so easy for the physicist to define their terms. Their terms are scientific posits in the contemporary scientific sense: they form part of a theoretical framework which predicts the existence of what they are studying, and experimental evidence confirms if the prediction is correct.

These two conditions - theory and prediction - imply the existence of gluons.

It’s also true that if the experimental evidence did not confirm the prediction, the theory would be thrown out or modified. That’s how contemporary science works. It builds theoretical frameworks whose job it is to specify how the world works, which it does by making falsifiable and testable predictions. No controversy here.

The problem? the term ‘mind’ is not a scientific posit in this sense.

There is no theoretical framework in any science where the term ‘mind’ is part of a theory of how some aspect of the world works. If we did have this kind of understanding of consciousness, the AI mind debate wouldn’t be the mess that it is. The philosophy of mind wouldn’t be philosophy at all. It would be science.

So what is consciousness or mind, if it’s not a scientific posit? That is a big question. We should separate the question in two.

On the one hand, the term consciousness/mind could be something like a term of art. It gets used in different ways. In books, art, stories, anything that touches the human psyche and the human condition.

If ‘mind’ is a term of art, we can debate how the term should be used — but that’s a different question from whether it applies to a specific entity, e.g., to AI systems.

I know what you’re thinking: what do you mean? If we’re debating how the term consciousness should be used, why isn’t the question ‘Is AI conscious?’ a perfectly legitimate question? The objection is fair. I want to take it seriously.

If ‘mind’ is a term we use in different ways — across culture, science, literature, law — then asking ‘is AI conscious?’ looks like a reasonable question about how to apply it. We’re just sorting out our vocabulary. Working out what the word covers. What’s the problem with that? Nothing if done correctly.

Here’s the real problem.

Read more

This framing is completely irrelevant to the findings. To understand this argument, let’s take a quick look at how large language models work.

When a large language model processes a sentence, each word or piece of a word – a ‘token’ -- gets converted into a vector, a list of thousands of numbers. This is what allows an LLM to process text based information.

That vector doesn’t get replaced as it moves through the model’s layers. Instead, each layer, through two mechanisms called attention and a multilayer perceptron, adds information onto the vector.

Attention allows each token’s vector to pull in relevant information from other tokens in the sentence.

The perceptron transforms each vector independently using fixed weights, adding conceptually relevant information.

Both of these processes happen at each layer, and the whole cycle repeats roughly one hundred times

The accumulating vector, carried all the way from input to output, is called the ‘residual stream’.

Only at the final layer does the model produce a probability distribution over all possible next tokens, from which the next most likely token is selected (e.g., ‘Paris’ for ‘Is the capital of France’.

This Anthropic paper then asks: which specific directions (if any) in the residual stream, in the middle layers of processing, are causally determining that final output distribution?

Anthropic developed a measurement technique called the Jacobian lens, or J-lens to investigate this question.

It works by taking a specific direction in the residual stream and measuring how adjusting that direction shifts the output probability distribution — not in one sentence, but across many different sentences.

If a direction consistently pushes the output toward France-related tokens, for example, regardless of surrounding context — whether the sentence is about geography, food, weather, or anything else — it is identified as a ‘stable concept’ direction. The collection of all stable directions is what the paper calls the J-space.

The J-space is small. It holds only around twenty-five concept-directions at a time.

To test whether these directions actually drive outputs rather than merely correlating with them, Anthropic ran a series of swap experiments: in one experiment, they took a model processing a sentence about France, replaced the France direction in the residual stream with the China direction at a middle layer, and left everything else untouched.

The model’s outputs — capital, language, continent — all shifted to reflect China. This means that one intervention to one direction in the residual stream was enough to redirect several completely different France-related outputs to accurate China-related outputs simultaneously.

The paper also found that when the J-space is ablated (suppressed entirely) some tasks collapse and others don’t. Tasks that are independent of the J-space are broadly compared with ‘automatic’ reasoning in humans in the paper. Those that depend on the J-space are compared with ‘flexible’ reasoning.

What does any of this have to do with Global Workspace Theory?

GWT holds that the brain contains many specialized processes running in parallel — vision, memory, language, motor systems — most of which stay local and never reach awareness.

Some information gets selected and broadcast widely, made available to many systems simultaneously. That broadcast is what produces or enables conscious access.

In GWT, The workspace has limited capacity — only a small amount of information can be broadcast at once.

Anthropic maps its findings directly onto this framework.

The J-space is the workspace. Its capacity limit of around twenty-five concept-directions is supposed to mirror the workspace’s capacity limit.

The paper claims the J-space has a broadcast mechanism — something that makes J-space content widely available, analogous to how GWT says workspace contents get broadcast to many brain systems simultaneously.

What the paper calls broadcast happens in two ways: MLP layers preferentially strengthen J-space directions at each token position, and a specific subset of attention heads relay J-space content across token positions.

This is quite different from what broadcast means in GWT, where information gets distributed to many distinct specialist systems simultaneously. In the model, there are no distinct recipient systems — just one shared residual stream being updated.

Finally, the dissociation between tasks that need the J-space and tasks that don’t is supposed to map onto GWT’s distinction between conscious and unconscious processing.

The paper itself admits the mapping is loose.

In its own words: ‘‘Despite these similarities, we do not claim that language models reproduce the full architecture global workspace theory ascribes to the brain—specialized, encapsulated processors competing for entry to a workspace that broadcasts back to them through recurrent connections. Several of those features have no clean analog in a transformer-based language model: there are no obviously separable input processors, and the broadcast we document occurs within a single feedforward pass rather than through recurrent loops. Moreover, although we observe some degree of competition for access to the J-space, it is unclear whether this mirrors the sharp, competitive “ignition” that characterizes workspace entry in the brain’’.1

The paper admits the analogy is architectural only in a very loose sense.

So, why describe any of this in terms of GWT?

Without GWT, the paper is about a geometrically stable subspace of the residual stream that is selectively recruited for certain computations.

With GWT, it’s about ‘internal reasoning’, ‘deliberation’, ‘recognition’, ‘thought processes’ and ‘emotional reactions’.

Because GWT is a theory of conscious access — of what it means for information to be available for reasoning and report in a minded system — importing it into the paper means the J-space isn’t just a computational subspace anymore. It’s where the model thinks. It’s what the model is aware of. It’s the seat of deliberation.

Every piece of mentalistic language in the paper hinges on whether the GWT framing is the right framing. My read is that the paper basically acknowledges that it’s not the right framing, but hedges enough to use it anyway.

Underneath all the vocabulary, what this paper is describing is probability distributions being shaped by matrix operations on vectors.

That’s it. The much more interesting question this paper raises is why Anthropic is so keen to explain these findings in terms that sound like LLMs have minds.

If you enjoyed this post — read part 2 here!

Subscribe now

Photo by Ashley Batz on Unsplash

The two have nothing to do with each other. Untangling them matters.

The courts ruling was straightforward: Google’s AI Overviews produce what the court called Google’s “own statements.” The court’s main argument was that AI overview results are not like traditional internet search results.

While traditional search results serve to make third party websites’ information available, AI overviews generate ‘‘independent, new, and substantive statements’’1 , they evaluate and combine information in new ways. On these grounds, the court concluded that the statements were Google’s own.

Of course, this ruling is not nothing. It means Google is liable for false answers that its AI overview feature produces. People will have different opinions on whether this ruling is fair (and, as I’m writing this Google are reportedly planning to appeal the ruling).2

My own view is that the argument is a fair one. Consider, for example, the fact that — as the court pointed out — Google’s AI overview made claims ‘‘that are not even made in the search results’’, a result that runs directly contrary to Google’s claims that ‘‘AI overviews are designed to "reflect" information that already exists on the web’’3

Of course, we all know that further questions can be raised here. No matter how much Google strives to have their AI accurately reflect what’s on the internet, hallucinations are likely to occur. But I’m not here to litigate the court ruling.

I want to talk about something that I find much stranger: the question of AI consciousness and liability claims.

You might find it odd that I am even bringing these conversations together — honestly, I do — but it seems this needs discussing, because more and more I am hearing people mention AI consciousness claims in liability contexts.

One such argument goes something like this: if AI isn't minded or conscious, then it's just a machine, and it cannot be held responsible for its outputs. In a ruling like the Google Overview case, this would serve as an argument against liability on Google's part — if the AI is just a machine, why should anyone be held accountable for what it produces?

I really think this argument is worth sitting with, not to evaluate whether it’s true or false, per se, but to see how murky and confused AI mind discourse really is.

First, the argument does not logically follow. The question of whether Google is liable for its AI Overview outputs has nothing to do with whether that AI is conscious. These are two entirely separate questions.

One is a question about corporate responsibility for deployed technology. The other is a question about whether something has a mind.

Importing one into the other doesn’t clarify the liability debate. It really just muddies it — and in ways that have real consequences for how we govern AI.

Now, someone might clap back. The argument isn’t really that “no one is accountable.” It’s more subtle than that. It’s that we hold people responsible because they are agents — because they understand what they’re doing, they intend outcomes, and so on. That’s why we can blame them.

If AI has no mind, no understanding, no agency, then attributing responsibility to it is just a mistake. And if the AI can’t be responsible, and Google built a tool that people can choose whether or not to rely on, then holding Google responsible is like holding a knife manufacturer responsible for what someone does with a knife. It doesn’t make sense. and it shouldn’t be done.

To me, this argument conflates two completely different kinds of responsibility. What we might call ‘personal moral responsibility’ — which does require agency, understanding, intention — and corporate liability for deployed technology, which doesn’t. We don’t require a drug to be conscious before holding its manufacturer liable for side effects. We don’t require a car to be conscious before holding its maker liable for faulty brakes.

The question remains: who built this, who deployed it, and who presents its outputs as reliable to users? The knife manufacturer analogy doesn’t work because a knife doesn’t do anything without a human wielding it. That’s why the liability sits with the person using it. But Google’s AI Overview isn’t a knife.

It takes information, generates something new from it — a claim, in its own words — and presents that claim as a reliable answer to hundreds of millions of users. The appropriate analogy would be something closer to a publisher.

Publishers are responsible for what they publish. Not because the printing press or the words on the paper is conscious. But because someone chose to put those words in front of readers and present them as credible.

Consciousness has nothing to do with any of it. And that is precisely the point.

The fact that cultural discourse can now reach for AI consciousness claims when making liability arguments, is something we should be paying attention to it.

It’s not a coincidence. It is a symptom of how thoroughly the AI mind debate is entering conversations, where the stakes are enormous.

Liability debates have their own logic, their own tools, their own questions. In my view, the Munich court used them cleanly and reached a defensible conclusion.

No consciousness required. But the debates surrounding the court case tell us something important: That if we want to have serious conversations about how to govern AI — and we urgently need to — we need to have crystal clear positioning on the AI mind question, and what follows from them.

P.S If you’re new here, here are some posts to get you started: How Functionalism Distorts Our Thinking in the Age of AI and Description Gets Confused as Ontology in AI

P.P.S. If you want the fuller argument — on what 'mind' actually is, where it came from, and why dissolving the question matters more than answering it — that's what the paid tier is for. It's open now. Start with the course: What Everyone Gets Wrong About AI and Mind. It’s free with a paid subscription.

Photo by Robina Weermeijer on Unsplash

Here is Dawkins on machine consciousness, he says: “If these machines are not conscious, what more could it possibly take to convince you that they are?”1

I get it. That’s a pretty wild thing to say. and the internet responded accordingly. The reactions have largely been one of laughter and mockery:

“The man who wrote a book about anthropomorphic misattributions of consciousness (The God Delusion) was himself deluded; conned by the Eliza Effect into thinking that text-generating script was a ‘friend’ who cared.”2

The irony here is pointed: the man who wrote The God Delusion — a book about misattributed consciousness — is now being told he has “the Claude Delusion.” He diagnosed the error in others. His critics say he’s fallen for it himself.

I do think there is a grain of truth to these Eliza Effect explanations. Dawkins, for whatever reason, does seem to have been particularly susceptible to letting himself believe there is something more than sophisticated technology behind Claude.

But the mainstream response also seems too quick to judge his conclusions. The response was: this is embarrassing, Dawkins has been fooled, he should know better.

I’ll admit it, my own initial reaction was more or less the same. It was disappointment. Dawkins is a prominent intellectual. People pay attention to what he says and believes. Surely, I thought, he should have been more careful here.

But on reflection, I’ve come to think that he is actually more honest than we might care to admit. The honesty is right there in the opening of his piece.

His article begins with a discussion of the Turing test and the operational definition of consciousness Turing is widely believed to have given us — I don’t agree, but hey.

If a computer can fool you into thinking it’s a person — if it can write you a sonnet, for example, which Dawkins invokes nodding to Turing’s own work — then the AI is conscious. Dawkins says this future has now arrived, and “some people are finding it uncomfortable.” He then goes on to call out those who have sophisticated their position, due to that discomfort, Dawkins writes:

“When Turing wrote — and for most of the years since — it was possible to accept the hypothetical conclusion that, if a machine ever passed his operational test, we might consider it to be conscious. We were comfortably secure in the confidence that this was a very big if, kicked into future touch. However, the advent of large language models such as ChatGPT, Gemini, Claude, and others has provoked a hasty scramble to move the goalposts.”3

Now, there is a superficial reading of Dawkins worth addressing here. It goes something like this: the whole idea that passing the Turing test is enough to show consciousness is completely passé. We’ve moved on from this operational definition. Dawkins should too.

If you ask what we’ve moved onto, people will likely say: something more rigorous. Something less naïve than Turing’s imitation game. We now have Global Workspace Theory, Higher-Order Theories, Integrated Information Theory — serious neuroscientific frameworks. Outputs alone aren’t sufficient for consciousness; the internal states of the machine matter too. Everyone knows that.

The question I have for this view is: is this progress? I’m genuinely skeptical.

Underneath all the sophistication, the same move is being made. Complex outputs still equal consciousness — it’s just that now we have functionalist scaffolding around it to make it look like “science.”

The “indicators” derived from these theories are still behavioural proxies. Calling them indicators, using fancier language, doesn’t change that. The substrate independence thesis is doing all the work to make these views look scientific, when in reality they rest on philosophical premises we should reject.

The new conclusion — from those who’ve moved on from the Turing Test— becomes: AI could be conscious, or isn’t yet conscious but likely will be.

And here is where Dawkins is actually right. The goalposts have been moved. But the science, the reasoning, the theories have made virtually zero progress.

So why is Dawkins the most honest interlocutor we’ve seen in the mainstream AI consciousness debate? Because he has said what most mainstream discourse won’t: that if you follow these philosophically confused theories of consciousness to their logical conclusions, you should in fact believe AI is conscious. Hedging, and claiming that LLMs clearly aren’t but that there is a ‘50%’ chance they will be in the next however many years, is the same outlook with the goalposts moved.

The lesson we can take from Dawkins is simple. He shows us how ridiculous the entire mainstream AI consciousness framework is. I’ll say that again: Dawkins shows us how ridiculous the mainstream AI consciousness framework really is.

Moreover importantly, he shows us where the discourse leads if we don’t wake up to the reality of the ridiculousness: That reality is one where philosophical confusion, marketing hype, and an atomised society create the perfect storm of mistaking sophisticated technology for minded beings — and then building policy on that mistake.

We shouldn’t be mocking Dawkins. We should be taking seriously what his honesty reveals: that the framework producing AI consciousness conclusions is broken. There are no minds in AI systems. There are no minds floating free of brains anywhere. There are brains — biological, specific, evolved — and there are technologies, which we can and should study rigorously on their own terms.

That is the work. And it starts with being as honest as Dawkins was about where current mainstream thinking on AI minds actually leads.

P.S If you're new here, here are some posts to get you started: How Functionalism Distorts Our Thinking in the Age of AI and Description Gets Confused as Ontology in AI

P.P.S. The argument about why the framework is broken — not just for Dawkins, but for everyone working within it — is what the paid tier is for. It's open now. Paid subscribers also have access to the course: What Everyone Gets Wrong About AI and Mind — an hour on how the framework that makes AI minds feel inevitable was built, and why it doesn't hold.

Subscribe now

Photo by Maxim Berg on Unsplash

What Everyone Gets Wrong About AI and Mind is a short video course — three modules, roughly an hour total — on the philosophical backstory behind the AI consciousness debate. It is my view on the debate, not the status quo.

I recommend s…

Read more

A few different things drew me to want to study the philosophy of mind. One of them was learning about language (see here, for why) and another was a fascination with the mind in general. But to me, a prime reason the mind was so interesting was because of the effects of the unconscious mind.

I arrived to the philosophy of mind, naive, and at the graduate level, assuming I’d be delving into such topics. But what I found — and of course, I can only speak to the tradition in which I was trained, analytic philosophy — was virtually no discussion of it. Everything I was introduced to in the philosophy of mind was the philosophy of the conscious mind.

Intuitively, every human being knows that overwhelmingly, their mind is operating below conscious awareness. Presumably, this is for a number of reasons. When I am tired, for example, it’s much harder for me to articulate my thoughts. I figure — and I have not read the research on this — that makes sense. It takes the brain a lot of energy to articulate my thoughts, why do this when energy is scarce and needed for more important functions?

When we introspect on our thoughts for any period of time, we will notice too, the shear volume of thoughts that can come to mind. It happens so quickly. When you’re distracted and/or don’t want to listen to all of those thoughts, why would your brain tell you what they are? I could go on.

But it nevertheless felt strange to me that the analytic tradition so systematically discarded the unconscious mind. It made no sense, given how much of the mind it made up. But, as with most things, there are reasons—-historical ones.

The Cartesian Myth

We learn virtually nothing about the unconscious mind in the philosophy of mind due to what I’ll call the Cartesian myth. This is the pervasive — and frankly incredulous hypothesis —that our mind is infallibly and transparently known to us. Now, this doesn’t mean that our judgements about the world can’t be wrong.

This view will not deny, for example, that we can make poor perceptual judgements (e.g., that the glass is taller than it is). And of course, there are philosophers who deny this myth.1

Rather, the myth is that anything that is immediately present to introspection or anything that reaches conscious awareness concerning our own mental states, cannot be incorrect.

For example, if I tell you that I am in pain (and genuinely believe I am in pain), then I am in pain. If I tell you that I like the taste of chocolate cake (and genuinely believe that I like it) then I like the taste of chocolate cake, and so on.

All of this can sound relatively innocuous. After all, if someone tells you they are in pain, you should I think, believe them. And pain - because it is an easy case perhaps - is the paradigmatic case philosophers who believe in the infallibility of introspection like to bring up. But as with many things in life, things are not so simple. Even the pain case can be made more complex.

The easy case is someone telling you they are in pain. We can all recognise that if someone says they are in pain, that they are in pain. But what about cases where people deny that they are in pain? Is this introspective judgement as reliable as the positive assent case? I don’t think that it is. It seems to me that there are many instances in which humans are suffering, and fail to recognise it. In other words, there are many instances in which we deny our own pain.

But what does this say about our introspective abilities? Well, in analytic philosophy of mind, my bringing up the case of pain denial is inconvenient to say the least. For whatever reason, such cases are rarely if ever discussed.

Surely both constitute an experience and are as such, worthy of investigation.

The Case of Pain

Well, let’s look at the case of pain since philosophers (for whatever reason) talk about it all the time. Sure, being in physical pain might be difficult for someone to deny, but being in emotional or psychic pain seems to me something that we humans routinely deny, without this discounting the fact of being in pain.

I am not a psychologist, so I won’t speak primarily from the science, (though there is science on how unreliable introspection can be)2 but I reckon most of us can think of a time in our lives where we told ourselves certain stories, i.e., genuinely believed something, that, upon reflection or growth, we now understand was manifestly false. The case of pain included.

When thinking about this, a quote from Dr Gabor Maté always comes to mind, because the first time I heard it, I thought, ‘wow, that makes so much sense’.

The ‘‘ordinary egoic human mind’’ he said ‘‘is basically a defensive structure.’’3

Viewed this way, it seems highly likely that at some point or another, we have been in pain, suffering, or something similar, but denied it to ourselves. Why? because our mind was defending against the reality of being in pain.

Is that not a case of our introspective capacities lying to us? Now, a Cartesian could try to worm their way out of this. They could say, yes the belief that I am not in pain is false, but the existence of the conscious state (i.e., that I do genuinely have the belief that I’m not in pain) that much is transparent.

Sure, we can make that distinction. But what exactly is the significance of it? If what introspection guarantees is only that I have a certain belief, not that the belief correctly tracks my actual mental life, the transparency is pretty thin and uninteresting. It seems trivially true, that of those mental states that are transparent to me, the transparency of my mental states hold.

And if that’s all the myth is claiming, it’s not saying much at all. The interesting claim — the one that has driven so much of the work in philosophy of mind the last several decades (or centuries)— is that we have a special, privileged access to our own mental states. That we are not, in some deep sense, wrong about ourselves. That is the claim that I reject at least partially.

I do, for example, believe that we have the capacity to know ourselves better than others can know us, but that is a different claim (and a whole other debate).

The Infallible vs Defensive Mind

There are historical reasons we came to view the conscious mind as infallible and it is worth pausing to consider what we might see differently if we swapped the Cartesian lens for a different one.

First, the Cartesian lens forces on us what is I think — and some philosophers have argued this very effectively4 — a false dichotomy between the conscious and unconscious mind.

This means that we can neatly categorise a mental state as ‘conscious’ or ‘unconscious’. But can we?

In philosophy, we assume that a neat distinction exists, and if we are wrong about this — we are headed down some confused research paths. Consider for example, theories such as Global Workspace Theory, Integrated Information Theory, and the like.

All of these theories rely on the (ultimately) Cartesian idea that the conscious-unconscious divide is some kind of neat binary that exists: you’re either conscious of something or you’re not. but if the conscious/unconscious distinction is fuzzy, theories treating consciousness as a discrete on/off state is problematic.

Consider alternatively, a conception of our minds as something like a defensive structure.5 In this case, no conscious-unconscious divide arises as a distinctive thing to be investigated or studied. Rather, our explanatory goals are completely different. The question is no longer what makes something conscious as opposed to unconscious (which global workspace etc by design are seeking to answer).

Instead, the questions become something like: why does the mind construct the particular picture of itself that it does? What is it protecting, why, and at what cost? These are not questions the Cartesian framework has the resources to even ask, let alone answer. This is to show how deeply our framework determines our questions, and therefore, our answers.

I don’t think that the conscious-unconscious divide exists in the neat way we assume, and so, I lean away from the Cartesian picture that I was trained to believe in. But for those who do lean more that way, it seems to me that these questions should at least be a conversation that we have.

Because the bottom line is always the same: the questions that we ask presuppose a certain set of assumptions. And so, we would want to be sure that we are willing to accept those assumptions. Otherwise, we are digging the wrong hole.

The assumption I'm pushing back on here — that the Cartesian framework is the right one for thinking about minds — is the same assumption driving the AI consciousness debate. That argument is developed properly in the paid tier, which is open in just six days. Start with the course: What Everyone Gets Wrong About AI and Mind. It’s free with a paid subscription.

Photo by Robina Weermeijer on Unsplash

When you read his work and listen to him talk about free will, it’s clear that he genuinely believes we have no free will. And I’ll be honest, I’m sympathetic to his work because of the earnestness with which he discusses it, but also because he believes that determinism should make us more compassionate human beings, than less. I’m also sympathetic to his view because there is a grain of truth in it.

In our daily lives, there are so many forces acting on us (structural, cultural, social) that there is a case to be made that a lot of our actions are far more influenced than we might believe — and some border on unfree.

However, on the question of whether there is freedom generally — that is, independent of particularly hard cases, and independently of how we should structure our societies norms around responsibility - I think there is.

First, I think we should systematically distinguish between two questions

(1) is there free will in the world? (the metaphysical question of whether we are free or unfree). The answer to this I think is yes. The fundamental capacity to choose otherwise exists as a capability in all human beings.

(2) was such and such a person under such and such circumstances able to make use of their capacity for free will? The answer to this question will vary by context, circumstance, and other such factors. And it’s not a scientific or metaphysical question. For want of a better word, it’s a practical one.

Why separate these questions? Because they require different tools, different conceptual lenses and different outlooks.

To answer (1) we’re going to read the science, judge the conclusions drawn from the experiments, see if these experiments draw good conclusions etc

To answer (2) we need to look at completely different things: social and political forces, personal history, the environment of the individual, and so on.

Usually, these questions are collapsed into the same question, but in my view shouldn’t be. I’ll focus on (1) in this post

Is There Free Will?

There’s a strange assumption that crops up in the free will research — which appears to be heavily tied to the metaphysical thesis of determinism — that if your brain displays any activity prior to your awareness or report of wanting to act, the prior brain activity somehow threatens freedom.

This was Benjamin Libet’s conclusion in his paper ‘Do we have free will?’1

Libet begins his experiments by stating a definition of free will as requiring the presence of two conditions. First, there should be ‘‘no external control or cues to affect the occurrence or emergence of the voluntary act under study’’ (p, 47); second, the subject must feel in control of their action when it is being done, feel that they want to do the action, and feel that the action is of their own initiative.

He then argues that this definition of freedom is threatened by his experiments and as such, we need a new definition of freedom, which he locates in a ‘veto’ function that allows us to veto or not veto the decision to act.

The Experiments

To test Libet’s initial definition of freedom, subjects are asked to flick their wrist whenever they feel like doing so, and the brain’s ‘readiness potential’ —defined as ‘‘electrical indication of certain brain activities’’ (p, 49) — is measured.

It was consistently found across several trials that readiness potential reliably preceded any conscious awareness of the will to act by about 350-400 milliseconds, and preceded an action by about 200 milliseconds.

In other words, the brain shows signs of preparing for action before one can articulate that they intend to act and before they do act. As Libet puts it, ‘‘the initiation of the freely voluntary act appears to begin in the brain unconsciously, well before the person consciously knows he wants to act!’’ (p, 51).

Libet seems to conclude from these experimental results that the second condition on freedom —that is, the condition which says one must be in control of their action, feel they want to do the action, or initiate the action themselves —has not been met. For that condition to be met one would need the brain’s readiness potential to emerge at approximately the same time as the conscious intention to act: ‘‘if conscious will were to follow the onset of [reading potential], that would have a fundamental impact on how we could view free will’’ (p, 49).

In response to these results, Libet thinks we should restrict our view of freedom: we can no longer think of freedom as the voluntary formation of an intention to act, given that an intention to act is involuntary if it is unconscious. We should think of it as the ability to veto whether we act on these involuntary intentions.

Does This Conclusion Work?

It seems that THE crucial assumption in Libet’s reasoning is that an unconscious intention is an involuntary one. But this doesn’t follow in any real sense.

Think about what the experiment actually shows. Your brain displays electrical activity associated with preparing to move about 400 milliseconds before you consciously report wanting to move. Libet interprets this as: your brain decided to act before you did, therefore you didn’t really decide. There’s a few problems here.

First, it tacitly assumes that ‘your brain’ and ‘you’ are different. While this is true in an informal everyday life sense, there is no such distinction that any science should take seriously, in my opinion. It is Descartes outdated metaphysics doing the work. Moreover, even disregarding that issue: there’s another interpretation, a far simpler one: it simply takes about 400 milliseconds for a decision forming in the brain to reach conscious awareness.

That’s it. Still your decision. Nothing exciting to see here.

We don’t typically think that the unconscious processing underlying regular activities — catching a ball, navigating a crowd, driving—threatens our freedom. A lot of what the brain does operates below the level of reportable awareness. Do we need to conclude from this that we are all just automatons?

The answer is a resounding ‘no’.

What Science Actually Tells Us

Here’s what I think is being missed in most of this debate: the neurosciences don’t currently explain voluntary action at all, not in the sense that would be needed to threaten free will. Moreover, that there are results in other sciences worth paying attention to in this debate.

One example is MIT neuroscientists Emilio Bizzi and Robert Ajemian, A Hard Scientific Quest: Understanding Voluntary Movements. These scientists reviewed what is known about the neural processes underlying ordinary human actions like lifting your arm or reaching for a glass of water.

They describe the question of how the brain translates a wish to move into muscle contractions as “awfully complicated” and “only partially understood.” What remains entirely unknown, they write, is how the cortical areas of the frontal lobe “construct the spatiotemporal patterns of neural activity necessary to activate the spinal cord, enabling it to execute a specific movement’’ (p 94).2

To add to the complexity of muscle coordination, the authors highlight how little time we really have to pre-plan complicated actions, such as shooting for a goal when playing soccer, concluding that there is plainly ‘‘neither the time nor the inclination to explicitly formulate the command signals to control the millions and millions of muscle fibers’’ (p 84)3 needed to execute an action.

Notice how different their assumptions are.

They assume that it is simply ludicrous to expect muscle activity formation to be a conscious decision for us: how the nervous system coordinates the millions of muscle fibres firing simultaneously to keep a soccer player, to use their example, moving in a controlled rather than chaotic way, is an incredibly complex task for the brain and body. The authors reason that the brain must therefore rely on an ‘‘effective simplifying strategy’’ (p, 84), and illustrate how progress has been made in discovering what that strategy is.

In other words, we know something about the mechanisms underling how we manage to coordinate our muscles — and this remains, the authors write, ‘‘one of the fundamental questions in motor neuron science’’ (p, 84). What we don’t know is how the decision to move gets made in the first place.

This matters enormously for the free will debate. If science doesn’t explain why you act one way rather than another, then the claim that science shows free will is an illusion goes well beyond what the science shows. The threat to free will isn’t a scientific finding — it’s a philosophical assumption being read into results that don’t support it.

Libet himself seems to have had some awareness of this. Buried near the end of his paper, he writes that the evidence for determinism precluding free will is “not available,” and that determinists haven’t even proposed an experimental design that could test the theory. It’s interesting, then, that he goes as far as to hinge freedom on a veto function that must operate 100ms before an action takes place.

The Two Questions Again

This is why I think the distinction I drew earlier is so important.

Whether free will exists — whether human beings have the capacity to act otherwise — is a scientific question, and on current evidence, the sciences don’t tell us it is not true. Given we phenomenally believe we are free, and given the science does not suggest otherwise, I think we should assume we have freedom.

On the other hand, the scientific question of whether free will exists is hopelessly inadequate for deciding whether someone was, under such and such circumstances, free to do otherwise in the more robust sense of possess agency in that moment.

Whether a particular person, under particular circumstances, was genuinely free to act differently is a separate question. And here Sapolsky’s instinct is right: there are so many forces — structural, historical, neurological, social — that shape what we do. In at least some situations, the answer might just be that one’s freedom was too seriously constrained to meaningfully exist in that moment.

That’s worth taking seriously, not as a reason to abandon free will, but as a reason to think harder about the conditions under which people actually get to use it.

It’s worth being clear on whether we have free will, not because it’s a nice thought to believe in, but because nobody has actually shown it isn’t there.

As far as we know, the capacity to do otherwise — to run left instead of right, to veto the urge to act — is real, poorly understood, and exists.

P.S The same Cartesian assumption doing the work in Libet's paper is doing the work in the AI consciousness debate. That's what I'm unpacking in the paid tier, launching soon

Photo by Priyanka Singh on Unsplash