In A Clinical Trial, Can Fitness Trackers Actually Track Fitness?
By Ed Miseta, Chief Editor, Clinical Leader
Many in the clinical space have been anxiously waiting for wearable devices to make it into clinical trials. The benefits they might provide to patients can be great, and we seem to be solving problems related to privacy and data collection. But how accurate are many of the devices currently on the market?
No one can question the current popularity of fitness trackers, including Fitbit Charge and Surge, and the Apple Watch, as well as devices from Samsung and newcomers XTC and Xiaomi. According to the IDC Worldwide Quarterly Wearable Device Tracker for the third quarter of 2015, total shipments amounted to 21 million units, up over 197 percent from the 7.1 million units shipped in the third quarter of 2014.
But are fitness trackers actually an accurate means of tracking your physical activity? Researchers in Japan recently tried to answer that question, by comparing the most popular devices with two research techniques considered to be the “gold standard” in the industry. The two standards are the metabolic chamber method, which captures all metabolic activity in a 24-hour period, and doubly-labeled water (DLW) method, which captures metabolic activity in a real-life setting over 15 days.
The 19 adults participating in the study were asked to undergo the two gold standard methods while wearing 12 fitness tracking devices. Energy expenditure was then calculated for each. An article recently published in JAMA Internal Medicine revealed a wide discrepancy in the results.
Total Energy Expenditure (TEE) was 2,093 calories for the metabolic chamber and 2,314 calories for the DLW method. This discrepancy is not unusual. Larry Husten, writing for MedPageToday, notes DLW is able to detect the effect of more activities than can be tracked in the structured-living setting of a metabolic chamber. But the TEE measured by the devices ranged from 278 calories lower to 204 calories higher per day. When compared to the DLW method, all of the devices underestimated energy expended, with the amounts ranging from 69 to 590 calories per day.
The conclusion of the researchers? Wearable devices do not provide a valid measure of TEE.
The Popularity Will Not Wane
Will this be a concern to the millions who are purchasing the devices? Probably not. The wearables market is not expected to shrink, although Gartner does expect the market share of fitness wristbands to shrink as consumers move to higher-tech devices like the Apple Watch and Motorola’s Moto 360. I got a Fitbit Charge for Christmas last year, and already own a Fitbit Surge. I also don’t think most consumers believe a wristband will yield 100 percent accurate results. The first day I wore my new Fitbit Charge, it said I climbed 37 floors, when I actually climbed about six. If a health-conscious consumer wants to know that they’re burning more calories, will it matter if your device records 2,500 calories burned rather than 2,400? Furthermore, if the difference in calories burned between the two gold standards can differ by 200 calories, should I be concerned if my wristband is off by 100?
Most consumers wear the devices for a variety of reasons: tracking daily steps, resting heart rate, distance travelled, and more. The motivation it provides to get up and do more is likely enough of a reason for many to use them. While far from perfect, they do provide numerous health benefits.
Of greater concern is whether these results will downplay the efforts to incorporate more wearable devices into clinical trials. Two additional studies, one appearing in JAMA Internal Medicine (a mobile phone blood pressure app) and one conducted the Scripps Translational Science Institute (using smartphones to monitor blood pressure, diabetes, and arrhythmias) also reported disappointing results.
And while measuring energy expenditure can be challenging, having a wearable track a patient’s heart rate while they walk to work, as opposed to coming into a clinic to walk on a treadmill for 30 minutes, is certainly going to be more user friendly. There would also be value to physicians in being able to see a patient’s activity level and blood pressure over a period of weeks or months. Regardless of the accuracy level of these devices, I would still put more faith in a wearable than in a patient diary that may have had several weeks of data filled in 15 minutes prior to walking into the clinic.
Great Insights For Clinical And Healthcare
Despite these findings, there is reason to be hopeful. Brett Villagrand, Senior Director of Business Development – Product Solutions at Oracle, remains hopeful. “We are experiencing a data and information revolution, and new technologies like wearables will continue to transform how we conduct research and development,” he says. “Mobile devices and wearables can offer great insights to clinical trial results and even more so in the healthcare space.”
Villagrand notes the concern over wearables has been that they rely primarily on accelerometers as their underlying technology. Even devices that have been "validated" for use in clinical research were not particularly reliable, according to the study. This can have the effect of creating some uncertainty amongst sponsors around the clinical utility of wellness trackers in a clinical trial attempting to monitor the mobility or performance of patients.
“This highlights the need for the industry to establish data standards for capturing data from wearable devices, especially if we want that data to be considered by both regulators and payers,” notes Villagrand. “While many companies are hesitant in their use of wearable mobility devices, they seem to have much more confidence in benefits that can be gained from medical grade devices that enable remote monitoring and measure specific outcomes.”
Villagrand believes there are still numerous arguments that can be made for the utility of wearables in clinical trials. Great insights can be gained, especially around patient centricity (recruitment, engagement, monitoring) and improvements to patient adherence are already being observed. In his view, there is a bigger impediment to adoption by sponsors in enterprise level clinical trials: the lack of a true business model or ROI for all parties involved. “Apart from the expenditure required upfront for the device (by the patient or sponsor), the infrastructure required for the collecting, storing, analysis, synthesis, and reporting of data requires a large initial investment, which potential clients are still struggling to justify.”
One company that has taken a lead role in demonstrating the viability of wearable devices in clinical trials is Medidata Solutions. In 2015 they even conducted a trial to prove that wearables were effective for use in clinical trials. The trial incorporated mobile health tools (a Fitbit, mobile app, and text messaging) to measure activity levels and quality of life for 20 patients with mild to moderate Type 2 diabetes. By all accounts, the trial was a success, especially for the patients who overwhelmingly had positive comments about the experience.
“We see tremendous potential for mobile tools to track patterns of behavior and uncover insights that cannot be detected in a traditional clinic,” noted Kara Dennis, Managing Director of Mobile Health at Medidata. “For example, we are working with sponsors who want to know whether their subjects are waking up in the middle of the night and walking around – the exact number of steps taken is less critical then the presence of movement.”
When it comes to data validity, the comprehensiveness, accuracy, and attributability, Dennis notes those factors are paramount. “The life sciences industry is actively looking for wearables and sensors that meet these criteria,” she adds. “However, measurement versus historical gold standards may ultimately be less critical than our ability to correlate this data with patient outcomes, endpoints, and other measures in a clinical trial.”