4:12
("Love" by Franco Gsiwvw) [JULES VAN BINSBERGEN] Welcome to the Lauder Institute at the University of Pennsylvania. I'm Jules van Binsbergen, director of the institute and a finance professor at the Wharton School. [JONATHAN BERK] And I'm Jonathan Berk, a finance professor at the Graduate School of Business at Stanford University. This is the All Else Equal podcast. Welcome back, everybody. Today we're going to look at something that has always fascinated me, which is in a competitive market, most economists would say that the competition is so strong that if firms don't correctly optimize, they get weeded out.
And so you would expect close to optimal behavior of firms. And I have to say, I buy into this. But today we're going to look at a case where it's, I think, very clear where you have a competitive market and firms are not optimizing, and that is the NFL. So before we get into it, let me first turn it over to Jules, because not every listener understands football, and Jules, I think, is one of them. [JULES VAN BINSBERGEN] You're so right about that, Jonathan. So just for our listeners in Europe, the NFL stands for the National Football League, and of course, in the United States, football means something entirely different than it does in Europe.
And so we're talking here about the sport that's called American football. And so the reason why indeed this is an interesting topic to study is that Doing an experiment in a firm where you really can observe all the possible decisions that can be made and where you have a good sense of what the competition really looks like is actually pretty hard. And so therefore. And our guest today, David Romer, wrote an academic paper that we are going to be discussing together. He essentially found a very nice competitive setting Where we can really rationally model what a competitive player would or should do, and then we can compare that with the actual behavior that we're observing.
And indeed, for our listeners to understand what we're going to be discussing next, it is probably a good idea to lay out the basic rules of football. So let me try to do that as in simple possible terms as I can, Jonathan, which is indeed not an easy feat. Now, for our listeners that are intimately familiar with football, we suggest that they skip the next three minutes and go straight to the five-minute and thirty-second mark. For those listeners that are not that familiar with football, I hope that the next explanation will be useful. So you have two teams. We have a hundred-yard field. And you score points by carrying or catching the ball into what's called the end zone, which is all the way on the
other side of where you start, so on the opponent's side. And if you get it there, then you can score in a number of ways. One way, which is the most spectacular way, if you will, is what's called a touchdown, where you touch the ball down on the other side, which is worth six points. And then after that you get an automatic extra point due to kicking the ball. So effectively you get seven points for a touchdown. Just for our European audience, think of American football, of course, as being much closer to rugby than it is to soccer.
Now, the team with the ball gets four attempts to try to progress on the field 10 yards towards the opponent. And each of those attempts are also called downs. And so it's important that in four attempts you make that 10-yard progress. If you keep on succeeding, then you get four new downs. And so eventually, if you keep proceeding 10 yards every time within your four attempts, then you reach the end zone the other side of the field. But if you fail and the other team takes the ball exactly where you were stopped, so the other team will take over the ball and will start from that point where you ended up in your attempts. So every possession is a sequence of, I would say, small bets. And the question that we're going to be really talking about
is that what is the best thing to do on the fourth attempt? So you've already failed three times to advance 10 yards. Now you're on your fourth attempt, and what should you do with the last one? [JONATHAN BERK] Okay, Jules, there's an important thing you left out. At any point, a team can elect to kick the ball through the goalposts. And if they manage to do that, they get three points. So there is a part of the field, usually from the 40-yard line onwards, where teams have a chance of kicking the ball through the goalposts. And so when the fourth down decision comes, they can go for it on fourth down, and then if they don't make it, they lose the ball
and the other team gets the ball, or they can try to kick the ball through the goalposts. And if they make it, they get three points, then the other team gets the ball. So now the crucial thing to understand is it's automatic to approximately 20 yards, if you kick it, you'll get 3 points for sure. But between 20 and 40 or 50 yards, it becomes less and less likely, and I would say at about 45 yards, it's impossible to get the 3 points. It's a crucial part of this thing. And the last thing I would add for somebody who's never seen the game of football is it's very much a possession game. When one team has the ball, it is extremely hard for the
other team to get the ball, so that who has possession is a big advantage. I think that's enough to understand the rules. So what our guest David Romer did was he used a technique which is ubiquitous in optimization and in economics called dynamic programming to calculate the probability from any yard line and any down Of expected scoring. And the first thing that is kind of surprising about the result of that calculation is that for yard lines that are very close to your own end zone, so let's say the one yard line, it's actually negative. The likelihood is the other team will score. And I think the reason for that is, first of all, there's a concept of a safety.
If they tackle you in your own end zone, the other team gets two points and the ball. But also, if the other team gets the ball close to your own end zone, Then the likelihood of scoring is very high. And so I think for those two reasons, on the one-yard line, you actually have a negative expected amounts of points. But of course that goes up, and when you're on your own 99-yard line, you have a very high expected number of points. What David was wondering about is the decision on fourth down. So the decision on fourth down is you could go for it. If you miss, the other team takes over where you are. If you make it, you get four more downs.
Or you could try to kick the ball through the posts. And then the last point was you could also give up the ball to the other team voluntarily by what's called punting. But when you punt, you kick the ball way down the field. And so the other team gets the ball, but on their side of the field. [JULES VAN BINSBERGEN] And when you kick the ball, Jonathan, you generally kick it 30 to 40 yards or something, so you set them back by quite a bit, but definitely not all the way. Well, depending on where you are at that point in time, you don't set them all the way back to the first yard. [JONATHAN BERK] Basically, any well-managed team is not going to find themselves inside the 20-yard line. So they will always take over at the 20,
and usually in football, somewhere between the 20 and the 40-yard line when they take over. Okay, so that's the decision people are facing. When should the team go for fourth down, and when should they punt or kick a field goal? And in football, when this paper was written in the early 2000s, teams almost always either punted or kicked a field goal. What David showed was that this decision was clearly suboptimal. The paper he wrote is really an outstanding paper. He thought about every possible reason why his result could be overturned and looked at it and did not find any good reason why
the analysis wasn't correct. Now, it's complicated analysis, but there are two cases where it's entirely obvious. The first case is if you've got one yard to go to get a touchdown and it's fourth down, he does a calculation and in expectation it's about the same. If you kick the field goal, they will get three for sure. If instead you go for the touchdown, you get seven. The probability of making it is about 40% or something, meaning in expectation it's 3 points. So from an expectation standpoint, at that point, you're indifferent. But David's point is that if you fail on fourth down,
the other team takes over on the one-yard line, and that, as I told you, is a negative number for them. [PARTICIPANT] Yep. [JONATHAN BERK] Whereas if you kick three points, the other team takes over on the 20, 30, or 40-yard line. And so you're indifferent about the number of points being scored, but you certainly aren't indifferent on where they take over. And because of that, it doesn't make any sense to kick a field goal in that position. And even today, we don't see teams do that.
[JULES VAN BINSBERGEN] So Jonathan, just as a clarification also for our European listeners, right, when we say 40 or 30-yard line, does that mean that from your side you've gone 30 yards, or does it mean that you have 30 yards to go? [JONATHAN BERK] You say you're in your own 30-yard line, it means you have 70 yards to go. If you say you're on your opponent's 30-yard line, then you have 30 yards to go. [JULES VAN BINSBERGEN] Yes. If you're all the way close to the end zone and you try the fourth down and you fail, the opposing team has to essentially 100 yards to cover, which is way more than an average play.
[JONATHAN BERK] Than an average kickoff. And so that is a big advantage, and since you're indifferent on the number of points, that swings it. [JULES VAN BINSBERGEN] That swings it. So that means that you should always try it. When you're really close to the end, you should go for the play, you should not punt, and you should not kick the field goal. [JONATHAN BERK] That's one of the obvious cases, and the other obvious case is if you've got fourth down in the middle of the field. So let's say you're fourth down at the 50-yard line.
It's too far to kick a field goal. But if you punt, I told you they have a kind of an option to start on the 20-yard line. So at most, you can get something like 20 or 30 yards with a punt. [PARTICIPANT] Yeah. [JONATHAN BERK] Okay? But if you make it on that fourth down, you are on the 50-yard line. And so that's another case where you should always go for it. And what happened when they wrote the paper? There was no change in strategy. But over the years, that strategy has changed. Today in football, teams generally go for it in that second situation. And so the question on the table is, why in such a competitive world,
with a strategy that's obviously better, Do teams not adopt the strategy? [JULES VAN BINSBERGEN] A number of questions come to mind immediately. The first one is, when we analyze organizations, the first thing that you and I always discuss, and we've discussed many times on the podcast, is what are the incentives of all the different players that are at stake here? And so there's a coach. The coach makes a decision, right? Is that fair? Is it to say that the coach is really the one who makes the decision on what happens?
[JONATHAN BERK] Yeah. [JULES VAN BINSBERGEN] So therefore, obviously the incentives of the coach could matter. That's one explanation. I actually think that there's a deeper explanation, and we'll see what David thinks of this, but Don't you think that the reputation of the coach also depends on what the audience thinks he's supposed to be doing, and that forgoing three points for sure by doing that play, and then the play fails, you look like such a fool that you want to avoid that scenario under all costs, despite the fact that in a dynamic programming sense for
the team and for the winning of the team, you would do better? [JONATHAN BERK] Yeah, Jules, I think that's certainly a possible explanation. It's stunning to me. The paper got a lot of press when it came out in the early 2000s, and the football teams knew about the paper. And they essentially ignored the paper. With the rise of analytics in general, they began to pay some attention, but as I said, it's still not fully optimal. They still did not go enough on fourth down. I think, Jules, this is the right time for us to introduce our guest today.
[JULES VAN BINSBERGEN] Our guest today is David Romer. David is the Herman Royer Professor of Political Economy at the University of California in Berkeley and the author of the standard textbook in graduate macroeconomics, as well as many influential papers in the area of what's called New Keynesian economics. The Nobel Prize was just given for that field. I think that in macro, David is very famous, but I think that in the world, David is probably the most famous for this paper.
This paper that ended up changing the field of football. It took a long time, but eventually I think he had major influence in this field, and this was probably not a field that David expected to have a large influence in, but I think it's a phenomenal study of optimal decision making under uncertainty, and I think a place where we can learn a lot about how firms operate too. [JONATHAN BERK] David, welcome to the show. [DAVID ROMER] Thank you for having me. [JULES VAN BINSBERGEN] It's great to have you, David.
[JONATHAN BERK] So, David, I may be South African, but I'm somewhat of a football fan. I understand the rules of football well enough to understand the basic insights in your paper and how incontrovertible they are. When you wrote this paper, what did you think was going to happen in football, and how did you expect the NFL teams to react to your findings? [DAVID ROMER] I had two predictions. Both of them were completely wrong. The first was that it would be hard to get teams to be aware of my research.
It's not part of the job description of working in the National Football League to read economics journals or working papers. Within months of when my paper came out, every team knew about it, and the channel I'd missed was journalists who are always looking for something new to write about, partly to drive readership and partly out of boredom. And so they all Found my paper and talked to coaches and so on. My second prediction was once they knew about it, of course they would change their behavior because they want to win games. And I thought the evidence was compelling, and that did not happen.
[JONATHAN BERK] This is the question that I think is the interesting question for our listeners and the Decision Making Podcast. You have a very competitive market, seeming incontrovertible evidence that the current strategy can be improved upon. And yet, at least immediately, you just saw no change in people's strategy. [DAVID ROMER] I think there's a prior question, which is maybe more important and more interesting, which was why before my paper weren't they doing this, right?
The arguments that Competition, competitive pressures drive firms to efficiency. They're so powerful, they're so logical. If you're not competitive, if you're a manager of a firm doing a bad job, you'd be fired. If the firm continues to do a bad job, it'll either be outcompeted or be taken over by people who can do it better. And that wasn't happening. If I'd uncovered some loophole in the rules that no one ever thought of, well, sure, that might have been something new. But you have to make a decision on fourth downs. People didn't always do exactly the same thing.
There was some variation. People knew it was a decision. There have been people making this argument Back-the first paper I found was from 1971. One of the authors was the backup quarterback of an NFL team. It was published in Operations Research. It was written up, I think, in Sports Illustrated. So our argument is not competition leads to efficiency if an egghead professor points out the right strategy. Our argument is supposed to be competition leads to efficiency, period. And that didn't happen. That wasn't happening even before I wrote the paper. And then it sort of adds to the puzzle that,
when I wrote my paper, which got a lot of attention, I think it had stronger evidence than what came before. It did get a lot of attention. I think that's a follow-up question. Why wasn't that enough to push it over? But I do think the prior question is the bigger one. [JULES VAN BINSBERGEN] So David, if we take a step back, how did you get the idea to write the paper? Was it already the case that you were watching games and every time you saw this happening, you were starting to get increasingly frustrated?
Or was it that you thought, let's see what comes out of this if I do the dynamic programming? Because only after you've computed the probabilities in all the different plays can you really make the case really strongly, right? So did you already know before you started that this was pretty much going to be the outcome, or was what you found quite surprising? [DAVID ROMER] It was largely what I knew. There are cases, and I walked through some of them in the paper, where you don't need the dynamic programming. You don't need the math.
You can do fairly simple examples, and it's compelling based on that. So I was pretty sure what I would find. The one thing I didn't know, I thought that If it's very short, it was fourth and one, fourth and two, the probability of making it would be very high, and fourth and ten, it would be very low. It turns out that's wrong. Fourth and one, I don't remember if it's 70% or 75 or 65, but it's not 90. And fourth and ten, it's not 5%. It's 30% or something like that. So, particularly in the part of the field where neither punting nor trying a field goal are very attractive, one thing that comes out is actually you should go for it with quite substantial distances to go.
That was unexpected to me, but the overall pattern was not. [JONATHAN BERK] So, David, in the subsequent years, and correct me if I'm wrong, I think there are two obvious points you can make without dynamic programming, without really understanding dynamic programming. One was what to do in what you call the dead zone in the middle of the field, and what to do with at fourth and one. Now, correct me if I'm wrong, the dead zone, the strategy did change, right, over the last 20 years? [DAVID ROMER] I think strategies have changed a lot over the last 20 years. So you just look at the fraction of times on fourth and one, fourth and two,
over the whole field that teams go for it, it's much higher than before. My impression-I haven't redone the analysis-my impression is they're not as aggressive as they should be, particularly in their own territory. It's very palpable that if you didn't make it, the other team would be in great field position. But there's been a very big change, and it's really been the last decade that's happened. [JONATHAN BERK] So a dual question. One is, what do you think eventually caused the teams to adopt a better strategy, given that they hadn't for so many years? And what about the fact that you very rarely see them going
in fourth and goal, which is one of the other points of your paper, that they should go more often there? [DAVID ROMER] I think in terms of what caused the change, I think Three things. One is the underlying forces that we sort of started with. Competition should produce efficiency. You wait long enough, maybe you will get it without anything else. The second's the general rise of analytics. So across all sports, including football, and the idea that they should Look at data, be analytical.
Every team, I think, now has an analytics, not just person, but staff. And you see that obviously in baseball, but you see it in basketball. I think you see it in soccer and just every sport. And then the third, which is kind of fun, is there was a precipitating event, I think, which is if you look at the data from before I wrote my paper till 2017, there's essentially no change. It kind of bounces around a little bit. Starting in 2018, there's a discrete step up, and there's been a trend ever since then to going for it more aggressively.
The intervening event was the Super Bowl 52. In February 2018, the Eagles against the Patriots. The Eagles went for it twice on fourth downs in unconventional situations. One was a very memorable trick play on fourth and goal. And the other, which I think is more important from point of view of my analysis, was they were down by a little bit late-ish in the game, but not super late. They had fourth and one deep in their own territory. They went for it. They made it. They drove down. They scored a touchdown. That was the margin of victory. As an aside, this led to a tweet that I just enjoyed absolutely enormously
from Bill Easterly, who's a very distinguished development economist. It was something like, "Who says that economists are useless? David Romer's been arguing for years teams should be more aggressive on fourth downs." Someone finally listened to him and they won the Super Bowl as a result. So that was fun. Anyway, it got a lot of attention, really got noticed. I think it's sort of teams woke up to the fact that, gee, it can make a difference, and so we should question what we've been doing and think things through.
[JULES VAN BINSBERGEN] So there are really, in my opinion, two interpretations of your paper, and also to put into context what happened after, I think there are two interpretations, right? So one is There is a lack of rationality or a lack of analytics, a lack of insight. People do not understand what they're supposed to be doing. As the technology progresses, it's possible for them to start optimizing. Once the optimization becomes easier, that's what they do. The second interpretation is, which I think is one way to read your paper, is an agency problem.
There's the objective of winning, and there is the reputation of the coach. The reputation of the coach is very much dependent on what other people think the right thing to do is, regardless of whether that's right or not. And once it becomes clear that the other people understand what the right thing to do is, that allows the coach to do what the right thing to do is, and therefore the agency problem, if you will, is resolved because the incentives are properly aligned. Should I read your paper more in the context of an agency problem that is resolved, or should I read it more as a behavioral phenomenon that data analytics has resolved for us, and therefore we get to more rational decisions, or some combination of the two?
[DAVID ROMER] I think my best answer is I don't know. Those are complicated hypotheses, and it's hard to get definitive evidence on them. A very common reaction to my paper from people who were persuaded by the logic was, Yeah, you don't want to be the coach who does this. If you make an unconventional decision and it doesn't work, And this is about- there's an upside and downside whenever you try this- you'll get pilloried and so on. It doesn't ring true to me, but this is not strong evidence.
It's really saying coaches knew, at least in the back of their minds, what the right thing to do was, and they chose not to do it. That just doesn't seem right, particularly if you think about the people who are coaches in the NFL, who have a lot of intensely smart, competitive, willing to defy things. The other thing about it is if you're coaching a bad team or you're a mid-level college coach who has ambitions of doing more, Doing something that defies the conventional wisdom and works, and on average, it's a good strategy, that has a huge payoff. So it's not just you did well; you did well because you were willing to buck the tide. So I'm skeptical of the agency theory.
The other reason I'm skeptical of the agency theory is, the pressure to do this- it wasn't the owners. Telling the coaches to do this. It was internal to the workings of it. And there had been people on the outside yelling at them again, you know, for decades that they were doing the wrong thing. So it's not like I gave them cover to do something that they knew was right. That's just not plausible. So I'm much more on the side. Again, I don't know, and I think that's very important, that their behavioral biases, they're very, very, very strong. They can block for decades these powerful competitive forces. But, you know, in some cases,
when the evidence is clear enough, things do change. So I guess I lean toward your technology explanation, but again, I don't know. [JONATHAN BERK] Although, David, the fact is, you know, I'm not that avid watcher of football, but I don't see them going fourth and one very often. I mean, normally they kick when it's fourth and one, right? [DAVID ROMER] So I don't watch much football either. I think I have to put in as an aside that something makes me uncomfortable about talking about football is that
it's a sport that probably shouldn't be played at all, given the costs it imposes on the players, and that sort of diminished my interest. But it's certainly clear they haven't gone all the way to the optimal strategy. I think that's pretty clear based on casual observation. I think what I don't know is whether they've sort of taken up The vast majority of the low-hanging fruit. So we talk about a deadweight loss from a triangle or an inefficiency from the triangle, and as you work down the bigger chunks of the triangle, even if you're not going most of the distance, you are getting most of the gains.
I don't know if they've gotten a large majority of the gains or just a decent chunk of them. I think someone would need to redo my paper with modern data and modern evidence about what teams do to answer that one. [JULES VAN BINSBERGEN] If the explanation that you're providing is indeed the technology explanation, would that also mean that you think that the arrival of AI could lead to huge productivity increases? Because if this would really help people get over all their behavioral biases and they're so prevalent, as your paper suggests, that must mean that introducing more rational decision-making should make a big difference, no?
[DAVID ROMER] I think for things like this, yes. But what's notable about this is it's a very well-defined decision. So that's why it was great to study. And it's also a reason that finding failures of optimization there is really compelling. This is a setting where you would be particularly likely to find optimization if it were happening, right? I mean, very clear decisions. The analytics are relative to how should we position our brand in the marketplace, or should we have an assembly line or team production?
This is a vastly simpler decision. I think for decisions like this, AI, absolutely. I could imagine AI getting to the point of, Dear AI, please improve the productivity of my firm or my team's winning performance. And it comes back with, "Here are 10 things that you could do." The bigger picture things, I don't know. Dear AI, how should we position our brand in the global marketplace? Is AI really going to know that better? It doesn't have a lot of data to go on. The payoffs are very long term. So I think that's a hard one.
[JONATHAN BERK] But David, what do you think the general implications are? I mean, I would take away from your study that in a very clear-cut, obvious case, we observe businesses in a highly competitive market not acting optimally. I would conclude from that, well, in a case where it's not clear-cut and the competition isn't as intense, you're going to see large departures from optimality. [DAVID ROMER] To me, that makes complete sense. Yes. So that's sort of what I was saying a moment ago. If they're not optimizing here, if they can't get this one right, there's every reason to think that
They're getting other things that are harder even more wrong. So that's one lesson. I think a second lesson is behavioral biases are really important. And if you think about it, the departures from optimality were in a particular direction. They were towards the cautious, the conservative thing to do. And if you look certainly other sports contexts, that's what we see. Teams were very slow to figure out that three-point shots were a good thing to be doing. In baseball, teams were much too invested in scoring one run at a time rather than playing for the big rally and the three-run home run. In hockey, my understanding is
teams are too slow to pull the goalie if they're behind late. So. It's behavioral, right? It's not there was randomly an error that they got wrong and then stuck with it. It's we're probably hardwired to be risk averse because, you know, being eaten by a lion is very, very costly, and that gets misapplied. I remember getting an email once from someone who coached high school who said, He went for it a lot on fourth downs, and the parents would come up and say, "But you know, it's a life-or-death situation." And he would say, "I fought in Korea.
I know what a life-and-death situation is. Going out for it on fourth down is not a life-or-death situation, and I shouldn't carry over the messages from Korea to coaching this football team." So behavioral biases are really big. I think another message is this question of why, given that the evidence was there, people were suggesting this, what is it about the organizations that made them, when it was sort of presented to them for decades, why didn't they respond? What makes an organization Have the flexibility, the openness, and so on to be open to the idea, gee, maybe we're making a mistake.
Maybe this professor at Berkeley, who's clearly never played a football game in his life, maybe there's something there worth looking into. [JONATHAN BERK] You know, David, the implication for the economy as a whole certainly must be that we're not doing a good enough job in teaching MBAs and undergraduates the importance of analytical insight, right? I mean, if it is a behavioral bias where people think it's a life-and-death situation when it isn't, the defense is an analytical approach, and yet we don't see that.
[DAVID ROMER] I agree. I mean, I think there are a couple of things that I don't teach in a business school, so I'm not going to criticize MBA education, but I think, yes, the value of analytics, but also the value of just figuring out ways of being open to mistakes. So I'll turn it into a research question. I have the sense that there's a body of research, certainly a stack of sort of top management books. I assume they're MBA classes on how to run, you know, a company in a way that makes it nimble, that makes it respond to new ideas, makes it overcome mistakes.
I would think it would be really interesting to sit down, say, with all those books and say, what are 20 hypotheses about what makes an organization be efficient, be optimizing? Is it about analytics? Is it about the structure? Is it about the culture, willingness to admit failure? And then say, how do those 20 hypotheses fit with explaining why no one did this for so long? What were the teams that picked up on it first, and so on? And I'd be fascinated to know if some of those hypotheses were really strong successes and others were not.
[JULES VAN BINSBERGEN] Because I think that also from a cultural point of view, it should not be underestimated. If I look at my own kids and their generation, what Moneyball, also the movie, what that did for the reputation of analytics within a sports context. Suddenly it's actually cool to do this, and I think it was a very motivating thing for young people to find out that using these tools could actually make you win a sports game. I don't think it had that reputation before, so I think that acceleration of that insight is good. So it may also just mean that we are still failing at marketing the tools that we teach, that they're actually way more valuable than what is coming across,
and it just requires Moneyball to really make that case from a marketing point of view. [DAVID ROMER] No, I agree. When I wrote the paper, I think I had a bunch of motivations in mind, and the journal editor who handled it really pushed me, I think, completely correctly in the direction of focus on this issue of do competitive forces cause firms to optimize? The editor who handled the paper was Steven Levitt at the University of Chicago, and his main comment was, "This is an intellectually serious paper. You should Take out anything that seems frivolous or cute or off topic and just focus on the core issue. And that was great advice, but one of the motivations was this is just fun.
This just shows you can use dynamic programming and linear algebra and things like that to get insights into a question where 99 people out of a hundred would think that's irrelevant. I recently went to my 50th high school reunion. And I would say half the people there knew about my work, about what I'd done on this. And rather than being the boring nerd who comes back, it was like, "Oh, there's this cool kid who did something on fourth down strategy." So that was nice. [JONATHAN BERK] Love it. But you know, what makes the paper so interesting is precisely as you point out.
It wasn't just a seat-of-the-pants analysis, and although you, in the introduction, there's a lot of intuition, really the analysis is applying dynamic programming in a very useful way. The problem with applying dynamic programming in a useful way is it takes a lot of effort. It's a lot easier just to do a seat-of-the-pants analysis, right? And so I wonder how much of the reticence is that you don't want to put the effort in. So it's easy to say, "Oh no, come on, that's bullshit. I know the game of football. I don't need to do dynamic programming."
The same as in a business. "I'm an expert. I've sold shoes for 30 years. I know how to sell shoes. I don't need somebody to tell me with a computer program how to sell shoes better." [JULES VAN BINSBERGEN] Although that cost has come down a lot, right? Particularly now with the arrival of new technologies, the cost of doing dynamic programming has become close to zero. [JONATHAN BERK] Maybe. Depends if you understand it. [DAVID ROMER] I think there are two issues. I don't know which one is right.
One is it's too costly, and so I'll do it the old way. The other is that means that if you're the person who's made the decisions by experience, intuition, seat of the pants, And to say, well, dynamic programming might do it better is relinquishing control, right? It's saying I'm not the expert here. And so it may be a psychological cost rather than a monetary cost, right? The cost to a football team of hiring someone to do dynamic programming is a drop in the bucket compared to what they do.
[JULES VAN BINSBERGEN] Agreed. [DAVID ROMER] One of the things we talked about beforehand was, how did teams react? And they didn't react by Adopting what I suggested. And regardless of how strong you find the evidence in my paper, the passage of time has shown that there's now a consensus that, for better or worse, I was right. Teams now, things have changed dramatically without a big change in the costs and benefits of it. So the way teams reacted, I think the closest that teams got to taking it seriously would be someone to engage in the paper long enough to identify something that was left out. And then the next step was, so we can ignore this.
Whereas I think if you had been reading the paper and you said something was left out, the next step would be, let me think about whether that thing that's left out, first of all, which direction it goes in, and second, could it be strong enough to overturn the results? And third, you know, how would I get evidence about that? And no team, as far as I know, got beyond the, "Oh, the analysis isn't perfect, therefore we can ignore it." That was the general reaction. [JONATHAN BERK] Well, David, I have to say thank you so much for coming on. I think this is such a good example, a clear-cut example of the limits of competition that I think it's an excellent paper. It really is.
[DAVID ROMER] Thank you for having me. It's been great to talk. [JULES VAN BINSBERGEN] Thank you, David. (piano music) Thanks for listening to the All Else Equal podcast. Please leave us a review at Apple Podcasts. We love to hear from our listeners. Also, be sure to catch our next episode by subscribing or following our show wherever you listen to your podcasts. For more information and episodes, visit allelseequalpodcast.com or follow us on LinkedIn. The All Else Equal podcast is a joint production of Stanford University's Graduate School of Business and the Lauder Institute at the University of Pennsylvania.
It is produced by University FM.
