Last week, I wrote that no AI model ever can attain consciousness because it doesn’t have a sense of its surroundings and its delimitations. In the physical world, one may call this a body.
This hypothesis brings us closer to agentic AI, because agents interact with tools that are not them, but part of the outer world. Agents thus need some elementary sense of what’s “them” and what’s “outside.” Therefore, agents could be a step closer to conscious AI — one step closer than simple LLMs.
One of our readers, Grant Castillou, pointed this out to me: neuroscientists have posited since 2006 that much more than just a body is needed for consciousness. In particular, there’s a short paper by Jeff Krichmar, which presents notes he took in 2006 when Nobel-prize-winning neuroscientist Gerald Edelman outlined a route towards what he called a “conscious artifact.” These notes remained pinned in Krichmar’s office for years and were finally turned into a short paper after a listener contacted him about a podcast.
It’s worth pointing out that this is not a rigorous research paper. It’s a presentation of lecture notes, and certainly not in itself an established framework. Nevertheless, it’s an intellectually attractive artifact to look at — as a reconstruction of a neuroscientist’s thinking about what constitutes consciousness and how the state of AI these days connects to it or not.
The Ten Commandments of Consciousness
According to Krichmar’s take on Edelman, ten ingredients are needed to build conscious machines, or “conscious artifacts” in their language. At a glance, these are:
Reentrant Architecture
Thalamo-Cortical System
Value Systems
Phenotype
Motor Control
Generalization in Learning and Memory
Communication
Thought
Language
Kindergarten
A. Computer Power
B. Minimum Neurons Necessary
Note: Any Sensor will Do But Need Haptics
What’s most interesting is that we’ve built the list from the bottom up so far. So I’ll discuss it from the bottom up, comparing it to where AI stands today.
Item B, artificial neurons, were first built towards the middle of the last century. Item A, computer power, has exploded in the past decade, which made LLMs possible.
Point 10, “Kindergarten” is some elemental training. Ask any AI engineer or machine learning (ML) specialist and they’ll tell you that training the model is the fun part of the job. Before LLMs, ML training was a big deal (I’ve trained many ML models myself). It’s been around for decades.
Language-capable models have been a thing before ChatGPT came around in late 2022, notably using natural language processing (NLP). However, chatbots have of course exploded this category in recent years. Note that this is number nine on the list!
We’re still grappling with how to make models think (or reason) properly. As for AI-to-human communication, we’ve made strides but it’s still an area of active research.
Generalization plays into the ideas of Artificial General Intelligence (AGI) floating around everywhere — tech bros acting worried that AGI will run the world and enslave humans, or some variety of such a dystopian calamity. Positively put, generalization is simply learning one thing through another, for example, an athlete being a decent hockey player with very little game experience, simply because they’re already excellent at soccer. We’re not that good, yet, at making AI models of any sort generalize in enormous ways, which is why AGI is for the moment not our central body of governance.
Motor control is tricky — robotics is making strides but remains a far cry from where we imagine it to be. A recent “breakthrough” was a robot capable of folding towels. I don’t know about you, but I can imagine robots doing slightly more complex tasks than towel-folding and floor-Roombaing. We still have lengths to cover on this point five of the list.
Phenotype, item four, is where it gets interesting. This typically refers to the physical design of a robot. Some robots look like sliding disks (like Roomba’s) and others resemble dogs or humans. As I argued in last week’s edition, it could also cover whatever an AI agent interacts with — tools, databases, servers, human users, and the likes. We’re still designing proper software environments for AI to live in. As for the hardware, there’s no obvious convergence to a proper standard — disks, dogs, humanoids, you name it, it’s all out there.
Value systems are not necessarily ethical guidelines — though pertinent and an active area of research. Value systems are what motivate a robot, such as salience or value of an environmental cue. For example, I’m mostly incentivized right now by the thought of having chocolate cake after writing this article. A chatbot might be incentivized by pleasing its human user to the maximum (which sometimes results in rather sycophantic AI, where the chatbot tells you that all your ideas are deep and original when they’re clearly not). Again, an active area of research.
Item two on this list is clearly from a neuroscientist: some kind of back-and-forth between a thalamus and a cortex. The thalamus regulates sensory throughput (like when you can’t hear your wife properly but your mother well — your thalamus is to blame), sleep-wake cycles and awareness, and memory. The cortex processes raw sensory data that the thalamus sent over, and it handles decision-making, speech, and voluntary movement. It’s the dance between thalamus and cortex that makes us able to act in everyday life, and Edelman posits that something like these two brain regions needs to be built to have artificial consciousness. I have yet to read about AI being built at scale following specific human brain areas and their interactions.
Finally, the reentrant architecture (first point on Edelman’s list) refers to dense, bi-directional neural circuits. Most neural networks only propagate in one direction. There may be some limited backpropagation also, but it’s nowhere close to a proper reentrant architecture. To quote Krichman’s paper, these global mappings of bi-directionally linked neurons give “rise to perceptual categories and action plans.”
A Body is Not Enough
My article from last week focused narrowly on point 4, phenotypes (as in, environments and bodies). Neuroscientists add to this some motivational reward (value systems, point 3), and some brain structures (points 1 and 2). At minimum — because most of the other points on this list are not complete yet either.
We of course don’t know if this ten-point list really is enough to built artificial consciousness. But it’s worth considering that, according to what a Nobel-prize winning neuroscientist said twenty years ago, we’re nowhere close yet.
Meanwhile, at Wangari
I’ll be off for the next few weeks! There are new things cooking, but I’ll keep those announcements for after my holiday break. You can expect to hear back from me on November 3, then again on the regular cadence.
Reads of the Week
Stop Building AI That Forgets — a very good, practical piece on the difference between an AI system that simply executes and one that compounds. Chandra Prakash and Raghav Mehra’s diagnosis is close to the question in this week’s article: conversation logs are not memory; a model should not be its own state manager; and a useful system needs to act when the state of the world calls for it, not only when someone opens a chat window. For anyone building agents, this is the infrastructure version of “kindergarten.”
The Complete Map of AI Scheduled Tasks — Jenny Ouyang makes the currently messy terrain of scheduled AI work legible: the border between a model and an automation platform, and where the proof of work has to live. This matters because an agent is not defined by the number of tools it can call. It is defined by whether it can take a bounded action, at an appropriate moment, with a record of what it did and whether it worked.
Protecting ourselves from 2 key risks of AI tool dependency — Karen Smiley has written a clear and humane piece about a question we tend to miss while admiring new AI capabilities: what cognitive muscles do we stop using when the tool becomes indispensable, and what happens when the tool is suddenly unavailable? Her answer is not to avoid AI. It is to use it deliberately, keep our own judgement engaged, and remember that dependence on a changing commercial service is also a business risk.
Ari, I enjoyed reading your article. I am currently exploring two questions in my research exploring (1) if AI systems construct and reconstruct knowledge through processes comparable to those involved in human cognition? and (2) are we limiting our understanding (and possibly development) of AI systems when we use human cognitive development as the standard against which we evaluate it?
I recently published an essay attempting to address these questions. Would you be willing to read it and let me know what you think? I’d love to discuss this topic and others related to AI with you.
https://jenniferhforrester.substack.com/p/jean-piaget-walks-into-an-ai-lab?r=1iu0cd&utm_campaign=post&utm_medium=web