15:55
[SCOTT BRADY] So, welcome everyone. So I'm gonna get to share a bit of background on AI GSB. So I've been back at the GSB for well over a decade. I teach a number of our classes, including our capstone entrepreneurial class, our research class, and then most recently as the faculty advisor, along with Brett Jordan, for AI GSB. But I think it's kind of interesting to share the context and the genesis of where this came from. So about 18 months ago, Sarah was preparing to be seated as the new dean, and Brett and I had just taught a new research class that was heavily based around applied AI. And I got that email, quite frankly,
as a faculty member, you never want to get. It's like, "Hi, Scott, I spoke to students about your class. I'd like to talk to you." Nothing good begins with that email. So I remember I called Brett. I'm like, "Did we do something wrong?" And he's like, "I don't think so." So I remember I sat down with Sarah, and what I realized is I did have it wrong. What I had wrong is she was excited. We were at this very formative point for AI, and she saw an opportunity for the GSB to take a leadership position and really wanted us to lean in and be experimental and figure out what we could do that would ultimately
help the GSB move forward. So what she did that was extraordinary is she found four of our incredible, what were rising MBA 2s that became AI scholars. And AI at GSB was an initiative that was the dean's initiative, so it was Sarah's initiative. It was faculty sponsored, Brett and I, but it was really led by an extraordinary group of students. And we're fortunate to have three of the four students here. So we have Jenny, Celeste, and Freddie, and Abby unfortunately couldn't make it. So I'm going to just walk through a quick narrative so they can share some of the background for how this actually happened. So I'm going to kick it off with Jenny. So you were actually one of the students that Sarah went and spoke with.
So one, I'm curious what she said, but when she was on her listening tour, what were you telling Sarah about AI? And in terms of your students, how prepared did they feel? [JENNI STEIGER] So, first of all, going to business school is always a huge decision. You're taking two years off in the middle of your career, and it's two years that you're out of the workforce, and you're hoping that you're gaining something in that experience. It's going to make you better off in the long term. And when we first got to the GSB, ChatGPT was just getting really good.
It just started to be able to do knowledge work in a way that it wasn't able to do before. And so we all had a moment where we were like, Whoa, it's going to be different when we go back to work. And so when I spoke to Sarah, I think a lot of the sentiment was that we were both hearing is that students were anxious. They were nervous about jobs. Are we going to have the skills needed to go back into the workforce? They were nervous because when you go on Twitter or look at the news, you're just inundated with all of these capabilities being released.
There's so much information about AI, and there was this Insecurity of, do I know what I'm supposed to know at this point? Am I up to speed? Am I scaling that learning curve at the same rate that I should be? Am I ahead of my peers? And for a lot of people, that was very nerve-wracking because they were learning AI on their own in their dorm rooms. And so they were looking for resources and an approach and help. I see one of my classmates here, David. You can get a testimonial from him later of how he was feeling and how he's feeling today. But it was a very nerve-wracking time for everyone, and I think
the sentiment was students were really, really hungry for something from Stanford. And so while it might have been a nerve-wracking time to go to business school, it was a great time to go to Stanford, and we just had to provide that. [SCOTT BRADY] So, that moment was interesting. There was both, I would say, in equal measure, intensity and anxiety. So in that context, how did those conversations shape what became AI GSB? Maybe specifically for everybody, what is the initiative? And how is it designed to alleviate that anxiety and really help students move forward quickly?
[JENNI STEIGER] Yeah, there was a couple things. So when we spoke to students, which was all of our classmates, it was really about, one, I don't know how to use the tools, or I do know how to use the tools, but I don't know if I'm using them properly or efficiently, effectively, whatever it was. Two, I'm not really sure about how AI is going to change my role and the industry that I want to go into. And then three, how do I make sense of what's going on in the universe of AI in the world? And so all of those things really shaped what AI at GSB became. And so what we wanted to do through AI at GSB was teach students how to use the tools, teach them how to set up their
environments and really be smart, efficient users of these capabilities, especially so when they went to job interviews, they could tell people what they built, show artifacts, and demonstrate that they had this capability. Two, help them really understand what was happening in industry, bring people in who were product managers today at big companies who could say, "This is how my workflow looked." You know, six months ago, and this is what it looks like today, and this is what's changing. So we had an informed view of what we were going back into instead of going blind. And then the third thing was bringing in thought leaders.
We brought in someone like Derek Thompson to come talk about whether AI is really a bubble. And for students who are going into investing or anyone who is going to be in a senior leadership position, you need to have a grasp on what's happening in the world and how it's going to impact you and your industry and your role and society at large. And so we tried to do those three things with AI GSB. [SCOTT BRADY] That's awesome. So, Celeste, you and Freddie were somewhat unique amongst your cohort. One, you came from very technical backgrounds, and you were also deeply, deeply immersed in AI. So how did you help to translate
Sarah's mandate into a program, and how did the students' needs shape the choices that you made? [CELESTE BEAN] Yeah, as all of you know, one of the benefits of coming to GSB is that it's very experiential. And so, as Jenny said, one thing that was really important that a lot of the students told us was that they wanted to be doing things themselves. And so that was the first thing that we built out. So this manifested concretely in a series of sessions where we were teaching people how to use specific tools, how to actually think about it intelligently as to whether or not something should be built with software, what are the implications of having a probabilistic model under the hood of whatever
it is that you're building, and making sure that they knew where to start. As well as giving them the tools to continue with whatever kind of creative ideas came to mind for them. So the first thing that we thought about was making sure that people actually knew how to use the tools themselves and that they had to put their hands on the keyboard. The second was, as Jenny said, making sure that we had practitioners come in who could talk about not only the mistakes that they were making, but also the way that they were fixing them. We think it's really important to hear from people who are responsible for what they're building, because you can hear about something, but it's very different to
make the mistake yourself and have to change it and make sure that it actually works. And then the third that we were thinking about was that people really wanted to make sure not only they were using these tools strategically, but happily, there was a lot of concern about how do we make sure we're using them ethically? How do we make sure that we're using them inclusively? How do we think about the broader implications of using these technologies? And so that came into kind of the third pillar of making sure that we were bringing in people who were kind of real leaders and who were thinking at a further horizon about what these tools might entail for not only our jobs, but really broader society, because we do see that this is a technological shift that,
I will not be the first person to say that this is a really meaningful technological shift. [SCOTT BRADY] So one of the really distinctive choices you made was having students teach classmates, which I think is pretty extraordinary. We all expect that, but not in the context of actually a classroom. What did you see happen when classmates began to take that role? [CELESTE BEAN] So there were a couple of things that were really awesome. So oftentimes we would start by having a session where, for example, Freddie or I might teach, how do you use this tool? But we really tried to teach, how do you be tool agnostic?
How do you think beyond whatever demonstration we're going through in this session? In that way, we deputized all of the students themselves to also be thinking about transmitting their knowledge to their classmates, which had a couple of benefits. The first of which is the best way to teach- or excuse me, the best way to learn is to teach. And so they had to think much more deeply about what it is that they were saying and making sure that they were grappling and thinking critically about the concepts that they had encountered. The second is that they kind of allowed for this movement into an identity where they could start to build things themselves, where they could take maybe an example that we had done in a session together, and then they could apply it
to wherever their subject matter expertise was. So, for example, I think of one friend who herself was an investor, but also really liked teaching Pilates. She, after going to a session, described to me the problem that she had of it's really hard to set up your music list, for example, for a Pilates class. I don't take Pilates very much, so this was fully in her domain, and I could basically say, "I think this is exactly what we did in the session. Go ahead and go do it." And then thereafter, her name is Amanda,
she was able to show other people what she had done, and this was someone who had kind of initially been thinking of herself as someone on the sidelines of what was going on, and who then had kind of moved herself into the action by virtue of having been deputized as someone who was allowed to have an opinion on this because she herself had built. [SCOTT BRADY] So, Freddie, maybe standing on the shoulders of what students learned, let's turn to what it actually meant for the work and career ambitions that they had. So what difference did you see AI@GSB make in students' careers, and can you give us a specific example?
[ALFREDO MÉNDEZ] Yes, you know, Scott, I think there was an identity shift. Like before all of these, the perception for technology, particularly for those who didn't have a technical background, was, "I don't belong here." Like this place is for engineers, computer scientists. I don't belong here. Then eventually, as people started to see that everybody can build now, and actually that their opinion matters, it's like, okay, I'm welcome here. My opinion matters. And even more, I think like last spring, for example, I remember I was taking this class with entrepreneurship and venture capital, the last time that Eric Schmidt was teaching it.
Eric was sharing with us that while he was the head of Google, there used to be this golden ratio of one product manager for 10 engineers, and that ratio now has shifted actually of two product managers for one engineer. So then now who's the bottleneck? And for many of these students, the people who used to be subject matter expertise, the people who have an opinionated vision of the world, now we went from like, "I'm not welcome in here," to "I am the missing piece." And I think that significantly also was reflected into the type of careers, industries, and even companies that people were aspiring for.
There were a lot of people, and I remember in the first week at Admit Weekend of like, "I come from this traditional background. I would love to go into tech, but I don't think I have what it takes," to like, "Of course I've got what it takes." And I think seeing that shift in us, first of all, our classmates was very powerful. [SCOTT BRADY] That's amazing. So how has the bar moved? So if we think about what's required to compete for the very best jobs that our students want, how has that changed over the past year or two? And how would you reflect on that?
[ALFREDO MÉNDEZ] It's competitive, and as you just said accurately, the bar has moved, and it's been moving fast. Let me set us in three different moments. Summer of 2024, when we started our MBA, the conversation, at least when we were talking or engaging with companies, was, "Tell me what do you know about AI?" Generally, having a good grasp, like these new technology, the transformers, how do they work? Next token prediction. Okay.
Especially if you didn't come from a technical background, that was great. Then fast forward to our summer experience, the question was like, okay, how are you using AI? Like, tell us and teach us. And at least for me, but also for many of us, we realized we were doing summer internships outside of the Bay Area, even internationally. I was, for example, doing a GMIX in Tokyo. We realized that when you come from a place like Stanford, when you are representing a place like GSB, the expectation is for you to be the expert, even if you didn't have that background.
And it was the companies that were telling us, and really relying on MBA interns to, like, okay, tell us, how should we define our AI transformation strategy? And you might imagine, but literacy was assumed and fluency was expected. And then, fast forward to spring of this year, when I was interviewing, I was interviewing for a fund with one of the most recognized AI researchers, Stanford professor. Now the expectation is to show me what have you built, and actually sometimes build it. At least my personal experience in the last round for this interview, it was the standard, okay, write an investment memo, tell us how would you analyze trade-offs,
write a product requirement document, very standard for a product manager. But then at the bottom of the instruction, it was: Build a demo, and not only build a demo, ship it, get customers, integrate customer feedback, and tell us what did you learn. And by the way, you have like 24 hours to do this. So I think like we really saw the goalposts moving significantly. But particularly, yes, I would say recapping, it went from like, tell us what do you know, to how do you use it, to show me what you've done and actually do it in a constrained amount of time. [SCOTT BRADY] I don't want to leave you with a cliffhanger. Freddy got the job.
Yeah. So maybe that brings us to what might just be the most important question is when building becomes easy, what is most valuable? [ALFREDO MÉNDEZ] I think it's a question many of us are still asking ourselves and it changes. If I were to sum it up in three things, and like alliteration, first it's conviction. Really having an opinionated but also well-informed vision of the world and the future, and being able to stand by it. Also say curiosity, really when everybody can be technical but not everybody's curious, and a lot of this is
learned by doing. Curiosity, I think that's incredibly, incredibly valuable. And lastly, also I would say, at least in my experience, what I've seen right now in the first months I've been having after graduating, Really what we learn here at the GSB, the human component, the interpersonal dynamics, the touchy-feely relationships, being able to understand what are the motivators, what to build, but also what not to build, and to try to be one or two steps ahead into what's gonna be the implications. I think that is really one of the places, at least I've been seeing, that all of the training that we get in here, that many times we wonder, is all of this leadership training, is all of this management,
is all of these interpersonal skills going to be useful? I think that really becomes more valuable than ever. [SCOTT BRADY] So I'll share one of the formative experiences I had in observing the extraordinary work that the students did. And I teach a lot of the exec ed we do here, specifically around AI. And I typically get two questions. One, oh my God, this is going fast. What's the first thing we should do? Typically, it's not about technology. It is about talent. The second thing is, what should I tell my kids? Which I think is kind of the most basic question. When things are moving so quickly and things are unstable,
what should I tell my kids? I actually thought Jenny hosted- she mentioned Derek Thompson- it was actually one of the most impactful events that I attended for reasons because it challenged my perspective. So Derek is one of the writers for The Atlantic. He's pretty apocalyptic, like generally speaking. You have anxiety after reading what he writes. So Jenny had a packed house. We're in one of the big tiered classrooms, and it is stuffed with students. It's late, it's hot, they're tired. And one of their classmates kind of got to the end in the Q&A, and she's like, "Fine, you've scared me.
What am I supposed to do?" And he actually gave the single best answer I've ever heard. He said, "I think you should treat AI like your health." And I thought, "Oh my God, where is this going?" And he said, "How many of you brushed your teeth this morning?" How many of you brushed your teeth this morning? Why'd you brush your teeth? Because you know having teeth 20 years from now is pretty important. How many of you watch what you eat? How many of you worked out? And everybody raised their hand. He said, "AI is not like the thing that got you in this room.
It was not an event. You went to the right school, you got the right job, you studied the right thing, and that's what enabled you to be here, and that was supposed to prepare you for the rest of your life." He said, "AI is like brushing your teeth. Every day, you have to get up and make a commitment to the future. And the future is this Pareto frontier of learning, this continuous process of continuing to learn." And then he looked at the students, he said, "Tomorrow, get up, brush your damn teeth." So with that, let's open it up to questions.
We have about maybe eight or nine minutes. These folks are extraordinary. They've experienced this in such an intimate and visceral way, and I'm sure they'd be happy to share their thoughts on their experience at the GSB. So maybe do we have runners if people have questions? Thanks for a great panel. Scott, you're being very open in terms of where the school is in addressing this. Could you address Jenny's questions? And particularly, how do you benchmark how well the GSB is doing in addressing these student aspirations and anxieties compared with other peer schools that the GSB aims to compare itself with and hopefully excel against?
[SCOTT BRADY] I'm going to guess they paid you to ask that question. Do one of you guys want to- I'll let them answer the first piece, and if they don't answer it, I'm happy to chime in. [JENNI STEIGER] Well, if you've seen the news lately, we were ranked as the top MBA program for AI learning. So that's pretty good. I think the great thing about the GSB is that the students have a lot of activation energy. They're like agentic themselves, you could say, and they do a lot all the time. And so now that we've really started this, we've kicked off this program, we've selected
a new set of scholars, which we got the same amount of applications for the next cohort of AI at GSB scholars as View from the Top in year 40 in our first year. So students, there is this hunger. Every single session that we ran was a packed room. They had to start giving us Obendorf and blocking it out for us. So students are really hungry. And so now that this program is up and running, the school is behind it entirely, the dean is backing it. We have a new set of scholars working directly with Scott on the programming and Brett Jordan, who I think is in the back corner over there, who are giving them mentorship.
The machine is up and running, and so I think we have the infrastructure to keep that going and keep getting better. I won't say that we've solved students' anxiety about AI. I think everyone is anxious about what's happening in the world. But I'll just echo what I said before. It's hard to go to business school, especially now, and make the choice to take years off. But it is an amazing time to be at Stanford, not only because what the GSB is doing, but we are doing some great stuff here with AI at GSB, but also across the street.
We have the best researchers in the world. You just heard from Andy. And so I think right now harnessing all of that is just an amazing opportunity. So as AI becomes more and more a fabric of everything. You're talking wonderfully about some of the things that you're trying to accomplish in specific initiatives. What would you like to see from the faculty, from the teachers broadly in the existing other classes that either that you're not getting, or what are you getting that you like to help Stanford continue to be ranked very highly among our sort of peer institutions? [SCOTT BRADY] Yeah.
[CELESTE BEAN] Two parts to that question. So one thing that we maybe didn't fully explore is that this is a fully student-led initiative for AI at GSB. And part of that is that a lot of the content that comes out of it is not evergreen. And so as it relates to the question that you asked, one of the challenges for faculty is that they come up with this excellent curriculum. It is not-I mean, maybe they can- but to change it every single time you teach a class, Right? Okay, I'm seeing enough nodding that I don't think I have to explain that further. That's a lot of work. And so I think that there's this confluence between the
benefit of having a super nimble organization like AI at GSB that leverages each incoming cohort's proximity to the technologies, right? Each of the subsequent classes is getting more and more technical. And so I think that is a great way to capture some of that information. From the faculty, it's been really interesting to see them be dynamic in a couple of ways. So one of which is making sure that we're using a flipped classroom. So for example, that you're using some of the synchronous time that you have together. To take advantage of the fact that you have all these smart
people in the room who have very different experiences and things like that. And so there's this move away in a number of cases from kind of an asymmetric lecture style, which I think is a great way to take advantage of not only these technologies, but also the benefit of having everyone in a room together. The second is that there's some changes that are happening on the evaluation side of things, right? So a take-home exam doesn't mean the same thing if we're just competing on who has the better Claude subscription, right? That kind of starts to lose its salience as a mechanism for reflecting how much someone has understood. And so there have been a number of creative mechanisms by which they have responded to that.
Perhaps on the less creative is going back to blue books, but which I'm personally a fan of. So you have to take the test in person and write it down. I see a lot of utility in that. But then the secondary is thinking about how do we make sure that you're responsible for the information, where it's moving more towards things like oral presentations. Which kind of goes back to what we were saying before about the best way to learn is to teach. And so, for us, even in preparing for this, right, we had to actually really understand what we were going to say if we wanted to have a cogent panel.
And so I think the confluence of having student-led, really dynamic content in conjunction with the expertise of the wonderful faculty who have been doing a very good job, I think, of trying to be quite nimble, has led to a really, I found at least, effective change. [ALFREDO MÉNDEZ] And if I can add also one more thing. Also paying a lot of attention to what happens inside the classroom. All of us in our year had to take one version of OSM, which is Optimization and System Modeling. I remember talking to wonderful professor Dan Iancu about what's been the impact of AI when all of these- It's like now systems optimization, you could
basically drop it in here. What he shared with us was very interesting. He was telling us definitely grades have increased, but especially variance across the grades has almost doubled or tripled, which means that, yes, we're all being pushed forward, but not evenly. And the people who brush their teeth every day, the people who are much more engaging, they are really going faster, orders of magnitude. And there are some people who, because of an identity shift or sometimes even access to the tools, then the gap between the ones who are engaging and not grows a lot. And I think that's definitely something that teachers and professors have been wonderful also paying attention to see
how can we address and support everyone wherever they need it. [SCOTT BRADY] I'll give you my perspective. So I'm fortunate I get to teach Irv's class, the one that Irv started, I don't know, probably close to 40 years ago, Formation of New Ventures, which many of you probably took. I took, my brother took. That really explores the arc of a startup. It was the quintessential case method class. Case method has evolved, and if you're at Harvard, it's incredibly rigid. Like, that is dogma. Like, you cannot deviate from it.
I would tell you the last two years has been maybe the most extraordinary experience I've had teaching. What we've been able to do is to say students, first of all, make them prepared to use the technology and say, use everything you need to do to be prepared to come into that class. And rather than use the historical case method where we have five questions that we pivot around, we teach it extemporaneously. We allow you to do anything you need before you go into class, and when you walk into class, there's no computers. There's nothing.
It's just you, your classmates, and the faculty engaged in an extemporaneous conversation. It's hard for us. I don't know where it's going to go, but neither do they. And we cold call. So now you're going to leverage these technologies to show up in the way you would in the real world. And I would tell you that the quality of that environment, the quality of the learning, quite frankly, the interest that it is for me to be able to be in there has just gone up tremendously. So I actually think the world post-AI gives us an opportunity to do something extraordinary. And I would say faculty across the GSB are largely experimental in how they're using it. So thank you guys.
("Another Horizon" by Pierre Perez Vergara)
