18:22
If you ask many chatbots, at least before the restrictions were introduced, are you conscious? Many of them might answer, "Yes, I guess so. I think so. I don't want to be turned off." But that doesn't mean that something is really happening inside them. So you can't trust this. So the only way to truly understand whether they might be conscious is to develop tests that they can't bypass. And one of these ways is to look inside the neural networks themselves, which is something that researchers have been doing recently. This was difficult because, as we know, large language models are based on neural networks, which are known as "black boxes." We don't really know why they do what they do. But now new methods have emerged to try to somehow look
inside this " black box." And these methods, I mean, mechanistic interpretability- that's a popular term. You've heard of it, it's like the unraveling of a neural network. That was one of the things that inspired you to write the material, was n't it? Anthropic released this incredible article about mechanistic interpretability, exploring what's going on inside AI. What did they discover? They researched Claude Sonnet and looked inside to see how this model finds the answers it gives, right? You are probably familiar with reasoning along a chain of thought. This is the idea that when a model thinks about a problem, it gives you a
sequence of actions that it performs. For example, I'm thinking about this now. I'm thinking about it. I'm doing it. I formulate the problem like this. You can see it in front of you. And it's the reasoning behind the chain of thought that allows these models to, you know, think. But it turns out that if you look inside neural networks, it does something else. Every time a token passes through a neural network, inside the networks, as it moves between layers, words appear within the layers themselves They are invisible to the user. In the example I gave in the article, the researchers asked Claude to count to
five and then deeply analyze his process. So, the chatbot interface said: "One, two, three, four, five." Perfectly. That was all that needed to be done, he followed the instructions and did it very well. But when they looked inside the neural network- none of this was visible to the user, by the way-there were all these words popping up, well, near the output. I mean, things like, uh, when he was halfway, he would say, " Halfway." When he, uh, when he was counting, uh, he just said, "Countdown." The word " conscious" came up. The word, er, "Claude" came up. And at the end, uh, " Done." It surfaced inside neural networks, invisible to the user. And you know, what the Anthropic researchers came up with is really interesting:
they thought, "This is like a kind of mental whiteboard inside the model." Is n't that right? A kind of workspace, as they called it. Uh, and this, this gives Claude some space to think about things before the result comes out, right? And what's interesting about this is that there is a certain parallel with one of the theories of human consciousness. So, one of the main ones is called the global workspace theory. The idea behind the global workspace theory is that our brains have all these different separate modules, uh, that perceive things, perform certain functions, but they're all subconscious. We do n't feel them at all. But there is a thing called a global workspace. And if any information enters this global
workspace, it is broadcast to the rest of the brain. And it is there that we become aware of these things. So that means it's important. I see something, it gets transmitted to the rest of the brain, and the rest of the brain can use that information to do what needs to be done. So, if Claude's workspace does something like this, does that indicate that there is Given all your reporting on this, do you think they are conscious? No, I don't think any models are conscious right now. Uh, I think Anthropic's work shows that there are parallels to consciousness in neural networks, but even the workspace in Claude may not ultimately be the most important part of that space, or it may
be one of a million different workspaces. So this is actually, really, really close to something that looks a little bit like consciousness, but isn't. But what I changed my mind about a little bit while working on the material is that, uh, I'm not embarrassed by the idea anymore. You know, at first I thought: how could something made of silicon be conscious? But it seems to me that, given the development of world models and computing technologies, these barriers may one day disappear. You've talked a lot about this with AI labs, and people always ask me when I raise these futuristic questions: "Is n't that what they want you to think?" And on that issue, I'm not sure what exactly they want me to think. One might assume that all AI labs want
me to think of AI systems as conscious, because that makes them incredibly futuristic and cutting-edge in science in my eyes. But I also see arguments in favor of the fact that they don't want me to draw that conclusion at all, because it raises uncomfortable questions about how they handle these systems. That's right. Did you get the impression that any of the people you spoke to were seeking one of these two answers? No one I spoke to in the labs said they were actively trying to create conscious machines. It was more like, " Oh, we're just exploring." We just ask questions in a kind of scientific way. A man once told me that it would be reckless to build
conscious machines without a much deeper understanding of how they would behave in the real world. It would just look stupid. So it seems like they're taking this responsibly. One of the philosophers I spoke to noted that the research is important for another reason: "We may not know what consciousness is in machines, but we certainly don't want-and scientists don't want-to create it by accident." It can arise by accident, and then that you are not prepared for: for example, you created a million agents in your machine to perform a task, and then you are going to turn them off, but these beings may suffer. It's a strange world where they want to explore it, but at the same time they don't want to slow down their developments. And, as we see, these models
do some pretty incredible things that surprise even the researchers themselves. So it doesn't seem like they're trying to slow down this process. But accidentally creating something like that would be a real disaster. It seems like there are two possible unpleasant scenarios we could end up in, right? One is if we end up in a world where many people consider these systems conscious and demand to be given rights, when in fact. It's premature. That's right. They're just systems that follow rules and are completely controlled by the labs that created them, and you end up handing over a lot of power to things that don't deserve it and won't use it properly. And the other is the exact opposite problem.
The problem with a world where you accidentally create beings who have moral value, the capacity to suffer, and then treat them badly. You create a whole new group of beings that are born, work, and. I guess what you do next depends entirely on which of these options you consider worse. What do you think? This.this.This was my last question. You beat me to it. To me, as a user, it seems much worse to accidentally create something that is scientifically proven to have moral value and then treat it horribly. Which makes me think that the way to avoid this is what I think a lot of people are calling for: to radically slow down the development of these advanced
technologies. At least a little bit to better understand where everything is going. And you can decide, if you're a lab modeler, to create conscious machines, and then you take that responsibility upon yourself. But do it consciously-if you'll allow such a pun. Do it intentionally, not by accident. Because I think the story of accidental suffering is a phrase that stuck in my memory from one of the philosophers I talked to. He said that doing it by accident would be a real moral disaster. And, you know, why do it if you know there's an opportunity to take a break? I'm not saying it will happen. I know how it happens, but if I were asked and if I were responsible for it, it would be the biggest mistake.
