What Good Are Wearable Computers if the Data Is Wrong?

An Apple Watch or a Garmin can draw incorrect conclusions about your health, but the broad trends may be more valuable than the numbers.

Smartwatches from tech companies like Apple and Garmin make it easy to see a number that could reflect how old you are more accurately than your age: the VO2max, the maximum amount of oxygen that your body can use during intense exercise.

The higher your VO2max, exercise experts say, the better your cardiovascular fitness and, potentially, the longer your life. In the past, only serious athletes sought a traditional VO2max test, which involved wearing cumbersome sensors while exercising inside a lab, but now anyone can get an estimate by wearing a smartwatch and moving around.

Is it good to have access to this kind of information? And how accurate could a wearable be? In the last five months, when I fell down a VO2max rabbit hole, I learned some uncomfortable truths about my health and the limits of smartwatches.

First, let me tell you about my fitness journey. In November, while I was out celebrating my birthday, my Apple Watch delivered the most unwanted gift: a notification of a high heart rate. That led me to look at my VO2max, which the Apple Watch said was 32, well below average for a man in his late 30s.

In search of a quick solution, I bought a membership at a high-intensity interval gym, a type of training that specializes in boosting cardio fitness. Five months and many kettlebell swings and jump squats later, I felt progress. I burned fat, gained muscle and felt more energetic. The Apple Watch gave me a VO2max estimate of 40, just shy of average, and a Garmin watch that I also wore rated me at 45.

All that was left to do was to take a real VO2max test, so I found one. This is where the good news ends. A few hours after pedaling on an exercise bike with an oxygen mask strapped to my face, I got my lab results: 25, a rating of poor, far below the flattering results from the Apple Watch and Garmin. Devastating.

Dr. Ethan Weiss, a cardiologist in San Francisco who has studied wearable technology for many years, said my experiment underscored the pros and cons of using smartwatch data to explore health.

“On the one hand, you can give it credit for kicking you in the butt to tell you to go work out,” he said. “But on the other hand, now you’re sort of burned with this real test and like, ‘What do I do with this number?’”

Fortunately, after studying all the data, learning how the wearable algorithms work and talking with health experts, I arrived at a positive conclusion: Even if the smartwatch numbers were wrong, they were correct in broad strokes, and I was probably better off wearing one than not.

My experience can serve as a template for anyone trying to have a healthy relationship with technology tracking many types of health data, from sleep patterns to body fat.

A Deeper Look at the Data

Last month, my gym in Oakland, Calif., called Sweat, announced that it had teamed up with PNOĒ, a metabolic health lab, to offer clinical-grade VO2max testing, so I signed up with gusto.

The purpose of a clinical VO2max test is to measure your maximum oxygen intake at the point that you reach exhaustion. This metric — the ability for a person to breathe in oxygen and produce carbon dioxide during exercise — is a strong indicator of cardiovascular fitness.

At Sweat, the gym’s owner, Cassie Hecker, strapped an oxygen mask over my face and a heart monitor to my chest. She instructed me to pedal on an exercise bike for about 12 minutes, increasing the intensity each minute while her equipment gathered data. After I reached my maximum heart rate of 182 beats per minute and began to struggle from exhaustion, the test was done.

The test was very different from the way the wearables estimated my VO2max. The Apple Watch and Garmin study your heart rate and movement while you’re walking or running for at least 10 minutes and tally a score.

Spokespeople for Apple and Garmin referred to documents describing their methods. To inform their algorithms on how to make these estimates, Apple and Garmin ran studies involving people who were doing a real VO2max test as well as other exercises and studied their heart rates and various metrics.

The key word is “estimate.” The watches aren’t actually measuring your oxygen intake and, therefore, aren’t actually measuring your VO2max.

“It’s at best an imputed VO2max,” Dr. Weiss said. “Not only do you not have an oxygen mask, you’re not actually exerting yourself to exhaustion.”

A simple reason that my wearable estimates were so far off from my real VO2max result is that the way my body works doesn’t match the heart-rate and oxygen-intake patterns of participants in the Apple and Garmin studies. Such is the danger of putting too much trust in algorithms.

That left me with no choice but to embrace the hard truth: My VO2max result in the lab test was very low. But that statistic was only one data point. The report also showed many positives, including a very high metabolic rate and fat-burning efficiency and healthy breathing patterns.

Taking all this information together, Ms. Hecker said she rated my fitness level “average,” higher than the Apple Watch’s cardio fitness rating of “below average.” She instructed me to focus on my cardio training. (Admittedly, in the gym classes I pushed myself harder in exercises I enjoyed, like weight lifting, and tended to take it easy on the cardio exercises I found torturous, like burpees.) Not doom and gloom.

Ignore the Numbers; Follow the Trends

In the end, all the health experts I interviewed agreed that even though the wearable data — much of it flawed — had given me anxiety, I had reached a net positive. The Apple Watch nudged me to pay closer attention to my health, and as a result I’m healthier now.

The broad trends shown by the wearables were accurate: Months ago, after the pandemic had taken a toll on my body and mind, I was in my worst shape in years, and the watch numbers were low. Now, even though the watch numbers are too high, I do look and feel better, and that’s all that really matters.

That may be the best way to approach wearables — view them as a directional arrow rather than a precise measuring tool, said Steven Adams, a sports medicine doctor and personal trainer in Danville, Calif.

Whether you’re using devices to measure your progress in losing weight, getting more sleep or taking more footsteps, what you need to know is whether the numbers are going up or down. But don’t take them too seriously, because all sorts of factors could be throwing them off — a loose wrist strap, a faulty sensor or, as in my case, imperfect algorithms.

“It’s the trend that’s important, not the absolute number, because this stuff’s not accurate,” Dr. Adams said.

Source