5:38
A dollar does not equal a dollar in the corporate environment. A dollar in the form of an abstract productivity gain is not the same as money that can be tracked through a business plan program and seen in the end result. That's why we believe that the tools we created ourselves cost about 10% of the cost of their commercial counterparts At the dawn of the generative AI boom, the board's mandate was simple: experiment at all costs. Well, the honeymoon is over and the AI bill has finally arrived. In the second episode, we ask tough questions about return on investment, tokenomics, and the real value of corporate intelligence. I spoke with two CIOs from the healthcare and financial services industries about the hidden costs of cloud
models, the " build or buy" debate, and how industry leaders are ensuring that the value of AI is driving revenue for them, not just AI developers. It seems to me that one of the key issues that arises both at the company and team level, as well as for individuals, is simply the issue of productivity. For example, how much can you do now? I wonder: if you make each employee 20-40 % more efficient, but don't change processes or headcount, are you creating real economic value? And how do you get that value if productivity remains at the individual level and doesn't extend to the entire organization? This is a very important question. I think this is the crux of the main question about payback that arises for many who have spent a lot of money on commercial
tools that become expensive very quickly. I'm sure we'll talk later about whether this is a good strategy; you can interview people, ask how much time is freed up. You will get quite large numbers. But can you point to the result? Do you see any of this in the financial report? In general, the answer is no, unless you have a very structured approach to capturing this outcome. In fact, there is implicit reinvestment in ways that are untraceable, and you don't know if it's valuable. So, everyone works a little better, does a little more. Everyone has a long list of things they've never gotten around to doing, and maybe they're starting to do them now.
How much of this extra work goes to waste? Who knows? So, this is not a managed reinvestment. I think there are ways to do managed reinvestment. But by default this is not the case. The default is to give people the tools to cope with noise . I have to provide an outlet for things that people find revolutionary and that they really need. This is one of the stressful aspects of the CIO job at the moment. But I also have to satisfy the CFO. I have to show him what we get out of this. True. In my opinion, the answer is that it should be cheap so that the choice is not difficult. Mhm. But being able to turn this implicit reinvestment of time into an explicit one requires thinking about how you will change jobs, leverage turnover change your
hiring structure, and so on. To make this a question where you have a real discussion about resource allocation. Yes. You know, ROI…I'm a former Wall Street employee. I worked at a large bank. When I came to healthcare, I realized that the definition of giving back was completely different. What is the return on discovering a rare disease that you wouldn't have found without AI? What is the return on identifying a tumor that you would not have noticed before the advent of AI in radiology? What is the return on a doctor spending time
with a patient because the conversation can now be automatically recorded using background technology? So, clinically, we are seeing huge improvements. We are seeing tangible progress- clinicians are asking for this technology like a person in the desert asks for water. This really makes a world of difference for them, for radiologists, etc. In cybersecurity, this is a force multiplier. We see the rate of attacks increasing, and we see them using good AI to combat evil. Regarding the corporate sector, I agree 100% with Paul. The evidence of productivity gains is not so clear. In the corporate sector, I would say that AI is very similar to our
transition from paper notes to email. No one really asked what productivity gains we would get from using email. You just have to use new technology, and it will give you efficiency, and then you'll look back and see, "Oh my God, we saved so many dollars." I mean, I'm sure you see this in my company too. AI writes a letter, then AI reads a replies to the letter, then AI uploads something, and you think, "What's going on?" And sometimes they send such long, long letters that I'm like, "Guys, can you just write me one line? What do I do with this?" Because, you know, sometimes I just see, well, too much of everything. But, you know, yes, I agree that the return on investment in employee productivity is not easily Well, I would describe it
differently, I guess. Because you can try to measure it. Now, you know, we did it. We interview people, ask them questions, calculate how much their time is worth and all that. And you get such a huge number that for us it's about 50 times more than what we spend on providing AI tools. So it's not hard to believe that it's worth it. But this is not the money. Isn't that right? A dollar does not equal a dollar in the corporate environment. A dollar of capital investment is not the same as a dollar of operating expenses. The dollar that goes into a is not the same as the dollar that goes into your income statement this quarter. A dollar that is a kind of intangible productivity gain is not the same as one that can be traced through a program with a
proper business case and added to the bottom line. So, it's not that it does n't exist, but that it's not the right money. And that's why I strongly believe that if we make it cheap enough-and I'm sure we'll talk about that later-we think that the tools that we've built ourselves cost about 10% of the cost of the commercial counterparts. This is a business case 10 times stronger than it would have been otherwise. And this ceases to be a problem. But if companies are spending hundreds of millions of dollars on these tools for their employees, that's a fundamentally different issue, because you can count every dollar. And the rest you can only see approximately, and you can't compare real money with virtual money. It doesn't
work like that, it's not like that at all Yes. 100%, tokenomics is a real thing. You know, if you give a developer a certain number of tokens, it 's hard to assess what exactly he spends them on. We were talking the other day about something like this: "Hey, if you're writing code to check the temperature outside, don't do it every minute. It doesn't change every minute. You can check it every 15 minutes, and you'll spend a lot less tokens." This is the type of thinking, because a token is almost like a credit card that is given to the developer, and the bill is paid by the company. And the developer does not feel the burden of this bill.
Maybe this person's manager, who is responsible for profit and loss, senses it. So I think you need to be smart about how you use tokens. Open source is definitely a more financially beneficial option. You know, a million tokens for Claude costs about $50, but for open source it's a few cents. So it's a great choice for low- criticality workflows. And for tasks where high accuracy is required, such as in clinical cases, or when the data needs to be clean, it may be worth using advanced models. But this field is constantly evolving. We are all learning, and we have time until 2030, right?
I don't know. Someone said something about 2030. Well, let's talk about costs. We are already talking about costs. So, are you now under scrutiny because of the cost issue? Do you get questions like, " Well, Rishi must have a magic money tree." " You have a magic credit card." This is true. Yes. No, we don't have that. We are a non-profit healthcare system. We're very frugal in how we. But we have a large computing cluster. We have our own, we manage our own supercomputer environment. It has a lot of graphics processors. It is called Minerva. And we can run a number of open source models there. We are not only talking about performance and cost, but also about data sovereignty. After all, you don't want to give away your data, especially intellectual
property. As you can see, OpenAI solved a mathematical problem based on someone else's previous work. We, as a biomedical research institution, do not want to transfer our intellectual property to these cutting-edge laboratories because we do not know what will happen to it. In addition, Anthropic is now literally engaged in scientific research. Yes, this is one of those things where you want to be sensible and make sure that all the knowledge in the world doesn't end up in just two cutting-edge models, because that wouldn't benefit anyone. Do open source discussion? Because for our viewers: open source models, technically these are models with open weights that can be downloaded from the Internet. You load your own data into them and run
them completely on your own. So they are mostly free to download, but you have to run them yourself. But it gives you a kind of Well, you don't have to run them yourself. You shouldn't. It's. It's true. There are many companies that will do this for you. Yes, our approach is that don't really have much desire to run models at all Okay, I understand. This is a headache. I imagine I envy your supercomputer cluster. But I guess it takes a lot of effort to get it up and running optimized. I especially don't want this headache.
There are companies that will do this very well. Yes. Our approach is typically to use cutting-edge models, but by renting them. We pay for each request. In fact, one of my biggest challenges right now is getting any cloud provider to give me the volume we use. I need 25 million tokens per minute for whatever model we use. To be able to meet our demand. Yes. Oho. So most of what we use now are queries to OpenAI, Anthropic, or Google models. We consider models with open scales. We have already used them on several occasions. This hasn't been a top priority yet, partly because the cost of smart services is falling so quickly that by the time you start to figure it out, you're already getting a 30-40% discount on something you
don't even have to think about for long. But we actually found that they work well in our tests. And this is part of my backup plan for cost optimization. Because I cut costs by about half or three- quarters wherever I go with an open-scale model of similar capabilities. And there are areas where we can, and almost certainly will. But there aren't many places where I can get the volume I need. M-hm. Because cloud companies actually have a huge problem managing peak demand across different models. It depends a lot on fashion.
Therefore, you will see huge fluctuations between different models, even day to day, depending on who posted what and where. And therefore based on sentiment on social media? Probably not completely. Okay. I hope not completely. To a large extent influenced by this. Wow. This is fascinating. And that's a headache I don't want to have. Yes. And I'm so glad Amazon, Microsoft, and Google have it instead of me. I think the interesting part about AI is that we talk about the choice between development and purchase. And we look at our organization, you know, we have to be smart enough
to understand what we can do and what we can't do, right? We are not a software development organization, right? So for commercial clinical products, we're seeing some companies like Epic, and even Microsoft Dragon, and some radiology technologies, offering really good use cases with AI already built in. You know, it's almost like. I don't think in a few years, or even now, there will be software that doesn't have AI. So you want to leverage the expertise of these people who are actually very knowledgeable in their field. And then you have your own research and intellectual property environment that you don't want to outsource. But I think eventually, maybe we'll see the cost of these models come down. There
