8:12
I'm Christian Chakovi, chief technology officer for systems design at IBM. Let's answer your questions from the internet. This is microchip support from the explain like I'm five subreddit. How are microchips programmed to know what is a one and zero? So, let's step back and talk about why zeros and ones are so important in computing. You can really encode all sorts of information as a long series of zeros and ones. Take the ASKI character set for example. It enumerates all the characters, the letters, special signs in a long list and then it assigns a combination of zeros and ones to
represent each of these letters. And then you can take a whole book for example and just string all the letters and characters and spaces and commas and exclamation marks to create a long sequence of zeros and ones. In a computer chip, a zero is usually represented by no voltage and a one is represented by some voltage like 1vt or 1.5 volts. Transistors in a computer chip can then modify the signals by either switching a transistor on or off and performing certain computations. You can build specific circuits using transistors like adders or multipliers. But you can also make it programmable so that you get software to run and perform ever more complex operations on these strings of data that you're sending into
the computer chips. Rafflecopter 4 asks just how on earth does a transistor physically work? Well, the easiest way to think of it is like a valve. You have an input and an output and a valve handle. In electronics with a transistor, we call the input and output source and drain. And the handle is called a gate. An electrical signal connected to the gate opens up the channel between the source and the drain or it keeps it closed so that no electricity can flow. On a modern chip like this, there are billions of transistors and of course they don't switch like very slowly like a handle but they can switch billions of times per second. Eat beef. Now why there are only few chip makers in the world? Well,
the modern chips are designed in very complicated processes. Like we're down to five, four, two nanometer design points and the manufacturing is extremely complicated. Building fabs that can manufacture chips like that is extremely costly and driving the development work to get to the next technology node from say 3 to two and from 2 to 1.4 is an extremely costly undertaking as well. So that's why we've seen a big consolidation over the last 10 or 15 years and we only have a few companies who can really afford to do the development work and build those fabs. So some of the big manufacturers today are TSMC in Taiwan, Samsung in Korea and Intel here in the US. All these companies are also building fabs
in the US and other places in the world, but that's where they originate. Venti 9 asks, why do computers get slow with time? Let me bust that myth a little bit. When you own a computer over a period of time, you're loading more and more programs onto the computer. You're getting firmware and software updates. You're loading more and more data on it. So, it's not that the hardware gets slower. You're just asking the hardware to do more because you're kind of accumulating a lot of junk on the device. It's not like the chips wear out and the hardware gets slower. It's just you're asking more of it. From Hatchphase, stupid question. Why do they need so many new data centers anyway?
Not a stupid question. With what's going on in AI over the last few years, we've really seen an explosion in new data center construction. And so, let's just step back. AI has had some really big breakthroughs over the last 3, four, five years, and it's really driving worldwide productivity for knowledge workers. Now, we spent trillions and trillions of dollars in wages for knowledge workers on the globe. And if it can make knowledge workers more productive by only a few percent that is a massive market, a hugely valuable market, many trillions of dollars worth. And so you see a lot of companies building data centers to capture their share of that market. Now these data centers are really complex. They need
enormous power supplies because they get filled with computers for storage, general purpose computing and then of course a lot of GPUs for all the AI processing. So while these data centers are really huge infrastructure projects with like big buildings and power supplies and cooling inside of the data centers ultimately there are millions of chips, memory chips, storage chips, general purpose processors and of course lots of GPUs for the AI processing. Since asking what are all those billions of transistors in my CPU actually doing? So transistors are really microscopic. A modern day transistor has only a few nanometers of size. A human hair is a 100,000 nanometers in width. So when
we're talking about five nanometers, that's like a tiny fraction of the width of a human hair. These billions of transistors, each one of them is a tiny switch. And we can form gates and gates or gates that combine two or three or four signals and compute the end of all these signals or the ore of all these signals, right? All four of them. one then an output is one or is one of them not a one then the output would be zero of the AND gate. Then we take these gates and form more and more complex circuits. We can build small adders. We can build multipliers. We can perform those computations in a loop. We can then add program functions so that you can actually program the chips and perform ever more complex computations.
And so because we're putting all of these circuits and multiple cores and a lot of memory on these chips, it adds up to billions of transistors. Minoi asks, "How can chips have billions of transistors but have very few external wires connecting them?" It really matters how much data you need to get in and out of the chip versus how much computation you perform on the chip with each piece of data. Take this chip for example here. These are all the connections on the back side. They are combining the power supplies for the chip as well as the input and output signals. And then there are billions of
transistors and a lot of memory on this chip to perform the actual computations. Dig Head and L is asking why do computer chips warm up? The computer chips consist of transistors and every time they do a switch there's a tiny current flowing through the transistor and the metal stack and when that current flows a few electrons get moved around and they push against the atomic structure of the metal and that creates friction almost as if your hands are rubbing together. That friction is causing the heat in the chips. from the ask science subreddit. If transistors are so small, like a few atoms, then how do we build them and put all of them on a CPU? We start in the manufacturing process with a blank
wafer. And then we put some photoresistive material on the wafer. We coat the whole wafer with that. And then a mask that has all of the fine structures of the design is used to shine a light on the photo resist. And then we edge out the areas that have been blocked from the light and we can deposit metals or we can dope the silicon and create the semiconducting properties. And then this happens in many layers. First we build what's called the front end which are the transistors using repeated steps of photoresist shine light on it deposit and then after the transistors are done with multiple layers we then put the metal stack on top which connects all of the transistors. Nowadays with the fine structures that we have the 2nmter
transistors it actually matters what kind of light we're using for that imaging. with today using extreme ultraviolet light because the wavelength of the light itself has to be small enough to even be able to show the fine structures that we need on these chips. The machines that do all that work are massive, like the size of an entire room because they need to super precisely position the wafer. They need to position the mask. They need to have the laser light positioned and all of that needs to be like really in lock step to be able to create these super fine structures of nanometer size. Then there are super fine machines that can cut the wafer into individual chips and we call
that dicing. And so you get these chips and then these individual chips are put on what we call a module. That's the little green board with two chips. And then there's a metal stack inside here that interconnects the two chips on this module as well as connects them to the underside where we have the pins that are driving the inputs and outputs of this chip. A Reddit user asks, "How was the first computer chip created with no computers to create it?" Well, really the first computers were designed by hand on a piece of paper. The circuits were drawn out on a piece of paper and then people would connect the different components with little wires that they would solder to the different components. I myself when I was at
Zaland University in Germany was in a computer class where we had what were called wire rep boards. You would put components into the wire repo on one side and you would connect little wires on the backside to interconnect all these components and we build a small calculator using that primitive technique. But in the 70s whole computers were built with these wire rep boards. Nowadays of course we have very powerful computers and we can use these powerful computers to build ever more powerful computers. We're using huge computer farms, for example, to validate the functional correctness of chips or to optimize the physical implementation.
Internal goal 955 is asking AI conquered software coding and hardware design is next. How do we prepare for inevitable displacement? Well, I don't really look at it that way. I think the word conquered is really strong here, too strong. AI tools are really powerful tools that make us engineers more productive. That's true in software engineering. That's also true in chip development and hardware engineering. But it's another set of powerful tools that we're building here that makes us better and allows us to build better chips going forward. I don't think it's going to displace us. It's going to make us more productive and enable us to build better chips.
Exile North asks, why is silicon so important in the manufacturing of computer chips? Is there any viable alternative? If not, why? So modern manufacturing processes for semiconductors in particular computer chips, cell phone chips etc are based on on silicon that is the technology that has evolved very far in terms of how many uh transistors we can put on a chip, how power efficient these chips can be, how we can manufacture them in a very reliable way. There are different elements in the periodic system that can be used as semiconductors. Silicon is one of them, germananium is another one. But for the most powerful computer chips, we're really dependent on silicon. A semiconductor is a material
that doesn't conduct electricity like metal does, but that can be configured to sometimes conduct and sometimes not conduct. That's the word semi. So you can build a transistor with the gate and depending on what signal you put to the gate, the semiconductor is either conducting or not conducting. That is the fundamental building block for uh modern chips. Rudabi asks, "What advancements are made every year that allow us to make faster processors?" A whole slew of things across the whole stack of chip development. The silicon node that's at the base, like is it a 5nmter, a 4, 3, 2 nm chip, uh, that changes all the time. Then micro architects like myself, we're inventing new ways to connect all these transistors and build faster processors
at a micro architecture level. They're figuring out how to make memory faster, how to make storage faster, how to make network faster. And in the combination of all those things, computers are getting faster, faster, and faster. A micro architect is one discipline in the broad field of computer engineering. A micro architect is somebody who basically lays out the big picture architecture of the chip before it then gets built into the different components and subunits that end up making up the billions of transistors. Dude with a bling asks, "Theoretically, how small can a microchip be fabricated?" If you go back to computers from the 1930s and 40s, they were built using magnetic
relays or vacuum tubes. Then in the 50s and 60s, we developed integrated circuits with the transistors on silicon chips. For example, in the span of my career, over the last 25 or so years, we've moved from over 100 nanometer transistors to 5 and 2 nm transistors nowadays. There's no really strict limit on how far we can continue to drive this. But there's research going on here, right? Nobody knows exactly how we'll build these chips in 10 years or 15 years because there's going to be some scientific breakthroughs. But let me tell you 15 years ago people didn't know how we would manufacture the chips that we have today with two nanometers. That was an unknown. So I believe we'll see the innovation continue and research
breakthroughs enable us to continue to shrink the transistors and therefore add more and more transistors on each of those chips. So we're now at 2 nanometers and we're entering really the research for the sub one nanometer time frame. We're calling that the angstrom age and we're really now talking about transistors of the size of just a few atoms. R202 asks semiconductor super cycle. Are we peaking or just starting? Crash coming? Well, who knows? As we've talked about, we're building massive new data centers and that is driving a lot of demand and it's really hard to build
additional supply for chip manufacturing just because these fabs are so enormously complex and expensive. So, what we're seeing is a demand surge from the new data centers and a bit of a supply crunch because it's hard to build more manufacturing fabs. How that plays out over the next few years is anybody's guess. Microchips have always gone in cycles. Memory cost, for example, has always gone up for a few years, gone down, gone up again. Right now, we're in what we call a super cycle. With all the construction of new data centers, there's so much demand for microchips, memory, processors, GPUs, and it's really hard to scale up the manufacturing capabilities because these fabs are so incredibly expensive that
we're really seeing a surge in demand driving the current cost of the microchips up. Are we peing? Are we crashing? That's really anybody's guess. I personally believe AI is such a transformative technology that this cycle is going to continue for a while. Doom Crystal asks, "If you can't put any more transistors on a microchip because the transistors are physically too small, why don't we just make bigger microchips?" There's physical limits to how big we can make chips, but then also there's commercial limits. The bigger the chip, of course, the more expensive it is. But let's talk about the physical
limits. When manufacturing chips, we're using masks to create the fine structures on the silicon wafer. And these masks can only be produced in a certain size. And so building chips above 750 or 780 mm, it's really hard. And those chips are already very large and therefore expensive. AIC a day is asking, what is the difference between a GPU and CPU? So let's step back. There's many different types of chips. There's memory chips. There's chips in a camera that recognize the light and turn the light into electrical signals, etc. A CPU is a historically very versatile type of microchip uh that is programmable and can execute all kinds of software. That's really the heart of your laptop,
for example, or the heart of a traditional server computer. GPUs are a different specialized kind of chip. They came about maybe 20 so years ago and really were designed for graphics used for example in either gaming or in applications like computer AED design. It turns out that the capabilities that you have in GPUs for most like real strong high performance computing capabilities are also very relevant to AI processing. And so the modern AI models have actually been kind of built around the GPUs because the math at a certain level is similar to the kinds of math that you do in graphics processing.
Programmer 7 asks, could someone explain all the different types of chip design engineers and the differences? Well, I don't think I can explain all the different types, but I can give you a good taste of it. It starts with the people who develop the silicon process, like how the silicon node and the chip manufacturing works. And then we have the engineers who design the chips. Starts with a micro architect who sort of lays out the big picture of how the chip should work. Then logic design engineers implement the different functions in the chip, the floatingoint units and the caches. For example, verification engineers make sure that
the logic design is functionally correct and produces the correct results when it computes on the data. Physical design engineers take the logic design and turn it into what we call a layout. It's really like where do which transistors go, which function goes where, how is it all interconnected using the metal stack. And then as the chip gets manufactured, you have all sorts of engineers and disciplines to actually put a system around the chip. So take this AI accelerator card. Somebody designs the card, somebody designs the module on which the chip sits, somebody puts it all together and validates it. And you have design for test engineers who make sure that the chip and the card works from manufacturing. So you have
all these disciplines that bring it all together and make sure that we have functioning computers in the end. Hyro it asks what were the tech leaps that make computers now so much faster than the ones in the 1990s. Really computer engineering has so many facets and everything gets better all the time. So it's faster transistors, smaller silicon nodes, it's better design in the processors themselves. It's faster memory, faster network, faster storage. Everything gets better. If you kept one thing the same as it was in the '90s, your computers today would still run very slow. So, it really takes all of it to come together in a full system design to create these breakthroughs.
Hooding complete 3081 asks, why does Moore's law keep ending every decade while computing power somehow keeps exploding? Anyway, Moore's law was postulated not really as a law but more as an observation that about every 2 years we can double the number of transistors that we can put on a chip. That law is still around. I mean it still kind of works despite it has slowed down a little bit, right? We're not doubling every 2 years, but we can continue to grow the numbers of transistors you can put on a chip. What really has broken down is the nard scaling. The nard scaling was a rule that you can make transistors smaller and smaller, put more of them on the chip and because of the transistors getting smaller, they end up consuming the same amount of power than the less
transistors in the prior generation. That scaling has really ended and as we're putting more and more transistors on, it's really hard to stay in the power budget for the chips that we're designing. And so that's why you're seeing, for example, processors consuming more power now than they did 10, 15 years ago. So with chip design now one of the most challenging aspects is how do we manage the power consumption of the chip with the dinard scaling no longer working as we put more and more transistors into a chip they consume more and more power and so there's a few key challenges here first to get the power into the chip and then that power creates heat and so we need to extract the heat and that's why you
see fans in your computers but that's also when you look at big data centers you see massive power lines go into the data centers and then You see cooling towers for example, they use a lot of water to cool the air in the data center or to even bring cold water directly to the chips to cool the chips with water. Bono Y is asking how are microchips made with no imperfections. I'll tell you the truth. When you're designing a chip with billions of transistors, there will be imperfections and we're designing to deal with the imperfections into the chip design. So for example when you're designing a memory element you are not just designing the say 1 megabyte of memory you're designing maybe 10% more
then you have switches inside where you can block out a bad memory cell and use a spare cell that we have put into the chip or think of some strange numbers of cores on a chip like you could have a chip with 28 cores for example well typically what you would find is there's actually 30 cores on the chip And then we look at which of these cores are actually working. And if only 28 of them are working, we can sell that as a 28 core chip. If only 16 are working, you could sell it as a 16 core chip. So we just need to prepare for that, have redundancy built in, and then structure the offerings so that we can also sell partial good chips.
United Nobody 2532 thinks putting chips in people's brains would be great. Well, let's separate what's actually happening today versus what might happen in the future versus some science fiction. We've put chips into the human body for decades already. Think of a pacemaker. The pacemaker is a microchip. It measures the electric signals in your heart and it recognizes something that is not working right and it can send a pulse to make the heart, you know, beat. Modern pacemakers also contain memory and take traces and are sort of like a ECG inside your body that can be read out at a doctor's office. We have these kinds of things. We have hearing aids. There's already research happening, for example, to have
artificial eyesight where a camera is connected, the microchips in the camera can be connected into the visual cortex of the brain. So we're seeing a lot of these things where I'll just say loosely we can mitigate disabilities or we could have situations where like you know a patient has a stroke and a chip could be used to repair certain sections of a damaged brain for example that already is happening and a lot of research is happening in that space as well. Where it gets a bit more complex and controversial is when it comes to actually enhancing the capabilities of the brain. To me, the brain is a finely tuned instrument that has emotion and intuition and experience and knowledge and it makes us think. It makes us be
innovative and makes us human. And I don't know whether, you know, putting an additional chip that could overload the brain with all the information that's out on the internet would actually help or hurt. It might just overload the brain besides all the ethical concerns it would create. A Reddit user asks, "Why does making chips require clean facility? Modern transistors are nanometers in size. A dust spec is thousand times that. Imagine as you're producing the chip that you have a dust spec settle on the wafer and blocks out thousands of transistors. Well, then the chip won't be able to work. That's why chip manufacturing facilities are super clean room so that you don't get the contaminations onto the chips that you're producing.
Data Nurse 47 asks those who develop chips, what was your career path like? Well, like in any industry, there can be many different career paths. Mine started as a computer science student at Siland University in Germany and then I joined the IBM development lab in Berlingan in Germany and I kind of learned chip design as part of my job. I then had an opportunity to move to New York and developed next generation mainframe chips and from there I kind of grew and went through different aspects of different chips. I designed processor cores, I designed caches, I designed IO circuits. I kind of moved around all sorts of different areas and then as my responsibility and frankly my experience
grew, I ended up in my current role as CTO. So I started as a computer scientist. Many engineers start with an electrical engineering background. As a computer scientist, I was more thinking in terms of how programming works and then I learned the electrical engineering part as part of doing my job. So those are all the questions for today. Thanks for watching.
