When trying to get the most accurate reading of energy expended during exercise, experts often look to tools such as the Metamax 3B.
Much like its sci-fi sounding name, it’s a rather convoluted contraption that includes a hi-tech mask attached to a computer with a volume sensor that can register various features of a person’s breathing, and effectively examine the reactions of the heart, lungs and the rest of the vascular and metabolic system while the body is under physical stress.
As accurate and validated as it is among researchers, it is not a consumer-grade product. And even if it were, it is not exactly the kind of thing you’d want to be lugging around on your morning run in the neighbourhood or at the gym. Not when you can just slap on a wristwatch that’ll not only tell you your heart rate, but also how many calories you’re burning.
However, the Metamax 3B was the chosen tool for a team of Canadian researchers looking to test the accuracy of consumer-grade wearable fitness devices for a study that was published in January 2022.
They examined the accuracy of the heart rate measurement and energy expenditure (calorie burn) of three high-end devices: the Apple Watch 6, the Fitbit Sense and the Polar Vantage V, and they compared their accuracy to the measurements provided by the Metamax 3B as well as a consumer-grade heart rate monitor chest strap, the Polar H10.
They conducted the study on 60 participants, split equally between men and women between 18 and 30 years of age, all physically active non-smokers, none with a body mass index over 30, and all white.
Heart rate measurement accuracy, skin tone and body weight
Although not explicitly stated in the paper, it can be safely assumed that the strict selection of white candidates is due to previous theoretical findings in numerous studies, such as this 2021 study, which found that heart rate sensors on consumer wearables are likely to be inaccurate on darker skin. Smart devices such as the ones mentioned above generally depend on optical sensors that continuously monitor the volume of your blood.
In between heartbeats, the volume of blood in your wrist will decrease.
However, unlike the heart rate trackers used in hospitals, which use infra-red sensors, the watch manufacturers primarily use cheaper green light sensors, with some using infra-red periodically. Green light has a shorter wavelength and is easily absorbed by melanin in dark skin.
The researchers behind the abovementioned study also found the sensors to be inaccurate for obese wearers, as obesity can affect blood flow and the thickness of the various layers of skin.
In fact, while they found that skin tone led to a less than 10% change in the sensor’s signal, a combination of dark skin tone and obesity led to as much as a 61.2% loss of signal from the watch’s sensors.
As for the other study, conducted on young, white, physically active participants with a BMI south of 30, all three watches – the Apple series 6, the Fitbit Sense and the Polar Vantage V – were found to be relatively accurate when it comes to heart rate monitoring across a number of physical activities including sitting, walking, running, cycling and resistance exercises.
They found that the Apple Watch was the most accurate for measuring heart rate across all five activities, whereas the other two varied on their levels of accuracy depending on the activity.
Calorie burn: Overestimated? Underestimated?
The amount of calories one burns during an activity are dependent on energy expenditure, which itself is highly dependent on a number of factors including weight, height, sex, age, body composition and the nature of the activity.
Based just on the variety of factors that affect energy expenditure, one can disregard the estimated calorie burn on gym machines. But what of the wearable devices that have far more data to work with?
Unlike the heart rate monitors which were found to be relatively accurate – at least for non-obese white people – the calorie burn measurements were found to be significantly inaccurate.
On average, throughout the different activities that were part of the study, the Apple Watch underestimated the energy expenditure by 103 calories per hour, the Polar Vantage by 63.6 calories and the Fitbit Sense by 130.8 calories.
Admittedly, finding out they’ve been burning more calories than they thought might be welcome news for those looking to burn more calories, but not so much for those trying to put on or maintain weight.
These findings with regards to the inaccuracy of calorie burn measurements are nothing new.
Numerous independent studies have tested a variety of smartwatches for exactly that, and come to similar conclusions, albeit sometimes some results point towards overestimation rather than underestimation.
Take, for example, this 2020 study which found that the Apple Watch overestimated the calorie burn 58% of the time, while the Fitbit overestimated 39.5% of the time and underestimated 48% of the time, and Polar devices overestimated 69% of the time and Garmin devices underestimated 69% of the time.
Simply put, while wrist-wearable devices may be relied upon for a number of features that are beneficial to our health – including apps that support health and physical activity – accurately counting calories burnt is not one of those, regardless of manufacturer promises. DM/ML
