Google's AI Health Tools Fitbit, Pixel Watch, and the New Health Coach App

Google's AI Health Tools Fitbit, Pixel Watch, and the New Health Coach App

Google's Senior Director of Product Management discusses how AI powers the new Health Coach app, Fitbit, and Pixel Watch to provide personalized health insights. The app consolidates data from multiple devices, including medical records, and features a conversational logging tool. The episode highlights engineering challenges, user testing, and collaboration with athletes like Stephen Curry to improve sleep and fitness tracking.

How We Use AI for Your Health | Made by Google Podcast S9E6. | Transcript:

- We realized, you know, working with Stephen Curry and other experts that athletes and others have a whole team of experts with where we've come with Gemini and we could actually offer this to everyone, helping them, you know, be their best. - Welcome to the Made by Google podcast, where we meet the people who work on the Google products you love. Here's your host Rachid Finge - There's a lot happening in the world of Google Health. There's a new app, there's a new coach, and there's a new device. So it's high time that we talk to the Google Health theme.

Here with us today is the Senior Director of Product Management, Andy Abramson. - This is the Made by Google podcast. - Andy, you worked on an unmanned helicopter before joining Google 15 years ago and then joining Google Health about two years ago. I'm wondering, is there a surprising similarity between working on a helicopter and working on maybe a device like a Fitbit Air? - Yeah, I think it's about complexity. Many people say that a helicopter is 40,000 parts that all need to move together in sync to keep a helicopter in the air. And in many ways, you know, the miniaturized technology all within a,

a tracker like a Fitbit Air, uh, all need to work together on your wrist and perfect harmony to, you know, deliver you health insights. - Quite a lot been going on, uh, on your team with Google Health. So I wanted to talk to you about three things today, of course, Fitbit Air and the Google Health Coach. But maybe let's start with the brand new Google Health app. For those who've never seen it before, how would you describe the brand new Google Health - App? The new Google Health app is everything you love about the Fitbit app, but redesigned and reimagined to give you a more holistic view of your health and to be the home

of the Google Health coach. So we know that many people use multiple apps, devices, tools to track their health even within Google. We have multiple of these apps today, but it's really too fragmented. So with Google Health, we decided we wanted to make one place for all of that data, uh, from Fitbit, from Pixel Watch, from all of your apps and devices that you use, including your medical records to give you that more holistic view, um, of your entire health journey.

- So if you say holistic, do you need a Fitbit or Pixel device like a Pixel Watch or Fitbit Air in order to use Google Health and get the data into it? - You don't to start using Google Health. This is the first time and part of the reason we wanted to rename it is it's not just for Fitbit. It does work amazing for Fitbit, let's be clear. But we're really excited that actually, you know, you can bring all this data, whether you're a Fitbit user who uses a Peloton or uses another strength app, you can bring all that data into one place.

But I think what you're alluding to is if you have, you know, an Apple Watch or another product, can you use it? The answer is yes. So, so for the first time we're actually reading from Apple Health Kit, that data from your Apple Watch and from other wearables, and we're trying to put all that data together into one picture for you as a user, which is really exciting. - I'm guessing that you knew beforehand that it would be a massive undertaking. So how do you then decide that it is worth it to actually pursue it? Yeah, - It's a good question.

Um, I mean, I think here at Google we often try to use a number of signals. So we use, uh, user research, uh, where we go talk to real users and show them prototypes. Uh, and there was a lot of that. I think the team started with ideas, you know, mockups and whiteboard drawings and quickly turned those into prototypes and we showed those to users. Users were really interested and um, you know, I think we as users of our own product were super interested and we knew that we were sort of on the right track and then, you know, kind of evolved from there.

- So one of the ways to get data into the brand new Google Health app is with Fitbit Air, a screenless tracker. We haven't seen one from Fitbit in quite some time. How did you decide to sort of go back to Fitbit's roots and, and create one without a screen? - I think around five years ago, the team started thinking about a broader problem. A screen adds a lot and you can see your data, but one of the things we noticed is that even though trackers and smartwatches, uh, have really taken off many people, more than 70% of people still don't have one.

Um, so we really sought to think about this broader problem. How could we build something really for everyone that would solve some of those problems? You know, for people who really wanted something that was like more comfortable or had a longer battery life and uh, maybe didn't look exactly like the smart watches that are out there today. Um, so the team spent, you know, years iterating, uh, to make something that was just so light, uh, and could solve some of these core user problems.

- Alright, and the end result, we both have it actually on our wrist right now. Um, so if you'd walk us through it for the people who haven't seen it before, like what is the device, what does it look like and and what are you probably as a team lead most proud of? - Yeah, so, uh, you've got one too. This is the Stephen Curry Special edition band. It's the easiest and simplest way to get a health tracker on your wrist. It's so light. Uh, it's our lightest, uh, tracker yet and it's made up of two pieces. So one, it's got the band and then it's got what we call the pebble, uh,

which is just the computer part. And these things just simply snap together. Um, and then you put it on your wrist and Velcro in it has, uh, 24 7 heart rate. It can measure, uh, your oxygen, uh, SpO2, your resting heart rate, all the things you would expect of a Fitbit tracker, all in this tiny little package. And one of the things I'm most excited about is really just how versatile it is. So, uh, we both have different colors on. Um, we know that like it is important part of people's fashion and I think the ability to just pop it out. So here I just popped it out of the Stephen Curry one and I've got a silicon one.

Uh, I throw this in my gym bag and I can just pop it in and in like, you know, two seconds, I have a totally different band. Uh, so we have some, uh, really elevated bands, all different colors. And to me that's something that's just totally different is how fast you can actually just change it, uh, you know, from workout to work. - I think what people liked also when it comes to bands is a while back, uh, Google actually released the specs of the band. So people could probably, you know, make their own. Why did we do that?

- Yeah. Uh, this was us leaning into something that really surprised us. Um, we were inspired by how much creativity we were seeing on, uh, on x uh, on Instagram. People were talking, uh, on Reddit and we listen and look at all of that. And we saw that there were a lot of users out there who were, for example, analog watches and they were, um, you know, experimenting. They were using our Fitbit bands. We were just looking at and fitting them to make mechanical watches or building custom bands. So we did something we haven't done before, which is we released the specs and we said, you know, it's not just the size

but the, the a tension to get that pop right so you can swap them. And the team worked really fast to get that out within a week and just kind of seize on like the interest and momentum from the users. And we're really excited to see what people come up with. - That's really amazing. So when designing not only the band, but also the pebble, um, did you have like any kind of particular goals or maybe even people in mind, uh, that Fitbit Air would really be awesome for? - I think really it was, you know, everyone I guess is sort of the answer your question, which is, which is maybe a not answer, but um, thinking about something that can work.

You know, you don't have to be technical, you don't have to want to have a smartwatch, you don't need all the apps. It's really, you know, you take it outta the box, you set it up in a minute and boom, you have a health coach on your wrist and that's really what we, we sought to do that it would be so comfortable that you don't even have to think about it. So designing for everyone was really the goal from the start. - So what struck me when I had it for the first time, and you already alluded to it, it's really small, it's super light.

There must be some tough engineering challenges to overcome when you try and create it in that way. - Oh yeah, absolutely. Like I said, I think they worked on it for more than five years to get the design right and a lot of those were pulling down, uh, the height so that the distance off your wrist, uh, is really low. As you can see, it's not only light, it's really small. So packing all that technology into, you know, this, uh, pebble and keeping it, you know, water resistant and you know, subject to all able to deal with all the daily wear that you can have, um, was really, you know, this is a really hard problem

to fit that all in there. And it took many iterations. Yeah, we were thrilled with what we came up with. - So when you're saying that you basically design it for everyone, that must mean that testing it, uh, is a huge undertaking in and of itself. Could you tell us a little bit about how you tested Fitbit Air before it came out? - We do a lot of testing on our team. I think one of the benefits of working on health and fitness trackers is we get to use them. We all, you know, do some amount of fitness in our life, so it's easy to test for us all.

Uh, although the flip side of that is we often end up with many devices on our wrists. Um, usually have a sort of, yeah, a whole collection. So we, we test it internally. So we got, you know, start with a small set of people, uh, to really figure out, you know, are the basics right? And then we expand it. Uh, so before we launch, we were testing it with thousands of people. Find these things that you don't find, you know, just from a couple people using it 'cause everyone's gonna have a different experience.

Um, and you get all kinds of crazy questions about the use that you would've never thought of. Um, and then a second thing we did is that we worked with our clinical team, our performance advisor, Stephen Curry and his team, um, as well as our consumer health advisory panel to get feedback from experts so that, you know, we're not only just deciding is this comfortable to wear, but like, you know, does it meet the needs for clinical and, and athletes as well. - You mentioned that panel. So what kind of role do they have in, in developing the product? - Uh, they're experts in their various fields. So, um, we pull them in, uh, not only to just test the product. So that's one of the first things we did when we onboard

people is we give it to them, uh, and ask them, you know, to use it and, and to give us any of their unvarnished feedback, uh, but also really to pull them in anytime we have questions because we're solving things that some of them have not been solved before. You know, how do you coach people on, on some of these really complex problems? Uh, when you develop fitness plans, there's so many different opinions from trainers around the world on the right way to approach things like, uh, progressing people and dealing with injuries. So, uh, we come up with ways to leverage these experts, uh, to help us make the right decision.

- Well, since you mentioned training, coaching, uh, there's the elephant in the room, right? It's the Google Health coach, uh, now available within the Google Health app. Talk to us a little bit about what the Google Health Coach sets out to do. - Yeah, the Google Health Coach is a personalized health coach built with Gemini. Uh, it's designed to help you navigate your personal health journey. So, uh, the coach can give you personalized insights, uh, adaptive coaching, and it's their 24 7. And maybe the way I would put it more simply is we realize,

you know, working with Stephen Curry and other experts that, uh, athletes and others have a whole team of experts there at their disposal, a medical expert, a sleep expert, a fitness trainer, a nutritionist, what we realize that some point is actually this vision that with where we've come with Gemini and the power of technology, we could actually offer this to everyone so that everyone could sort of have this team of experts, you know, in their pocket on their wrist, they're helping them, you know, be their best. - So how do you start building that?

You mentioned it is built with Gemini, so it has a great foundation already. Where do you start? - So the team worked really hard on something that we launched as a lab, uh, two years ago, three years ago maybe, uh, called Insights Explorer. So, um, we kind of started at some of the hardest problems and said, you know, hey, how do we take the data and, um, Gemini and put them together and really see how it's gonna work? So we launched a lab to Fitbit users and we asked users to try it out, ask all their hard questions. We learned a ton from it. It was not a complete product, it was like a subsection of the app. And then, um, we launched a preview actually, um, which you might've seen, uh, where uh,

we learned a lot from our users about, you know, what they were looking for with a product that really hadn't been built before. - Yeah. So what, what was your own experience with Google Health Coach? You've seen it grow up, I guess, in a way and getting better and better. Do you maybe remember a first time that it gave you an insight, you were like, wow, I didn't know that about myself, for example?

- Oh yeah, I think there's many. I've been trying to become a better runner, probably like many people I find running hard, it's difficult. Uh, I'm not very good at it. I don't run very far, uh, but I aspire to be a better runner. So I recently tried to get into trail running and I decided I would make that my goal with a personal health coach. You know, I was able to tell it what my goal is, uh, just in natural language. I said like, I'm not good at running and I wanna become a better runner.

Uh, so I worked with the Google Health coach to actually, you know, have a plan where I can like increase my running every week. Uh, but really I was asking a lot of questions like I don't really understand running cadence and, and like what is good running form and how often should I run? All of these things are things I could work with the health coach on. Uh, and I guess I'm happy to say I'm definitely a better runner.

It's still hard, uh, but I know way more about it and I'm actually getting out there and enjoying it. - Well done. Good job. What I hear from a lot of people is that, and it's something that struck me as well, is that the Google Health coach is maybe one of the few, if not the only sort of AI based health coach that is super proactive. It does more than just sort of tell you in a different way what your stats are, right? - Yeah, this is definitely one of the things that we sought to solve from the beginning. So we have on the Pixel Watch a feature called Morning Brief.

And every morning it's not AI based at all every morning it just presents this data to you in a way that's really useful. You know, like, how well did you sleep? How, what's your readiness based on some of your vital signals? Like her rate variability? This is one of our most loved features on the Pixel watch. Uh, so we sort of saw this and we were inspired to what, you know, this is actually what a coach should do.

A coach should actually, um, you know, every morning have you cut through the noise of all this data and tell you what, what really matters. You know, how well did you sleep? What should you do today? So this key insight around like moments in the day that matter, which is like in the morning when you woke up after a workout at the end of the day, at the end of the week, really like these are moments that like a real coach would interact with you and just tell you what you need to know. So we've used these as anchor moments to really design the product around.

- You mentioned Stephen Curry a little bit. Um, what was his role in creating the, uh, Google Health coach? I know he was instrumental in creating one of the bands for Fitbit Air. You have it on your wrist, but what about the coach itself? Like it was he, was he a tester as well, for example? - Yeah, Stephen was a tester, uh, as our performance advisor. Uh, and he has a whole team, which who are great. His personal trainer, Carl also is basically part of our team. Um, and we talk to him, you know, sometimes every day he's an early person, early bird and he would be, he know texting

before I'm even awake, you know, problems with the app ideas. Um, and this was great because, you know, Carl and Stephen have all this expertise on how you train a professional athlete. Um, you know, Carl himself is a, uh, like a world renowned fitness trainer, but like training an NBA athlete and learning what they do, you know, there's so much we learn from them and so many ideas that we want to take forward in the product in the future. And, you know, Stephen also, uh, you know, it's not just about fitness as a, uh, one thing I learned is that a, as a professional athlete, he's on airplanes, traveling between games and actually sleep is a really difficult part of his journey.

You know, he's often like catching naps to catch up on sleep and talking to the press until really late and then get getting home really late. So actually we talked to him actually surprisingly a lot about sleep. Um, and maybe I can share one fun fact that one day he actually, we heard like the sleep algorithm's not working, he's not getting any sleep and you know, building a product that's changing all the time, like we often have bugs, but we've looked into this bug and we found that actually that algorithm did not detect his sleep because his vitals were so good as a professional athlete, his heart rate, everything, uh, it didn't think that he was a, he was asleep.

So we actually incorporated his data and other professional athletes' data to improve the algorithm to catch these vast outliers of people who, you know, are really pushing, uh, pushing the edge. - Amazing. I, I'm, I, I'm afraid I'll never end up in that tier. So the algorithm works perfectly fine for me, but it's amazing to, to see that how you, how you adapt it and actually, um, that is something that, that I noticed as well. The health coach. It's not like, you know, it launched and you're done. You seem very, very busy in continuing to improve it wherever people, uh, see a need for it.

- Yeah, we're definitely trying to do something I think a little bit different here. You might have seen, we posted a roadmap. Anytime you launch a big product, there's always tons of feedback and we are really excited to see that feedback and we wanted to not just like listen to the feedback and then go back to our offices and work on it. We actually wanted to like tell users that they're heard. Um, so yeah, me and my product team were definitely reading social media, we're like talking to our support team.

Uh, and yeah, we shared a list of near term improvements. Uh, and we're actually trying to keep this up to date and we plan to share, you know, much more with users in the future because we think this is how you'll design a better product, is actually just listening to users, conversing with them. And then, you know, back to our point about building for everyone, if we're really gonna build for everyone, we have to listen to, you know, as many people as possible. - Alright, so Andy, what will be your pro tip for anyone? Well let's say they download the Google Health app, right? Because maybe they already have a tracker or they just got the brand new Fitbit error.

What would be your pro tip to try out in, in the new app? - Yeah, my pro tip is what we call conversational logging. So it's logging like, you know, saying you ate something or exercise. We have a whole set of things you can log in the app. And what we did is we used Gemini to process camera documents, things like that. So for me, this is kind of game changing. I do strength workouts and I never keep track of them. Um, I know some people are very diligent, they have like notepads.

Uh, what I've started doing is actually go to the gym. Uh, if the, if there's a workout on the board, um, I can just snap a picture of it and I say, log this act this, it keeps track of all the weights. And if there's not even on a whiteboard, I can just say it conversationally. I can just use the microphone button and say, I did three sets of this, six sets of this. Uh, I did two minutes of core and um, it all goes into my workout. And the cool thing about this is actually this will create a much better coaching experience.

'cause a coach can keep track of like how much weight I'm lifting if I'm progressing. And that's really one of the problems that we need to solve is like, there's only so much you can measure from the wrist. You know, you can't measure exactly how much you lifted. And a real coach needs to know that information. So yeah, pro tip is try logging stuff. You can try logging your food, uh, no more like exactly looking up the nutritional value on the back. You just snap a picture of it. Oh, wow. And say, I had this for breakfast.

- I did not know I'm okay. So tonight I'm definitely gonna take a picture of the whiteboard after the training and, and see how that goes. That's a wonderful tip. And as well for the, for the food logging. That is great. Andy, it sounds like you'll need to come back to the Made by Google podcast at some point because you are so hard at work. Uh, I'm sure you're gonna come up with something even cooler in maybe the next year or so. So please come back and tell us about your progress.

- Yeah, we would love to, uh, you'll, we'll probably be back sooner than you realized. We have lots of great, exciting things in the pipeline and, um, yeah, we hope to keep hearing feedback from you and, uh, all of our users. - Thank you for listening to The Made by Google podcast. Don't miss out on new episodes. Subscribe now wherever you get your podcasts to be the first to listen.

More Tech Transcript