The apparent inaccuracy of wearable fitness trackers has long bugged us, but we had been unable to put real scientific numbers to it. Fortunately, there are real researchers in the world, and they’ve established scientifically that most of the step-counting data you get from trackers is pretty bad.
A letter in the Journal of the American Medical Association — as mainstream a medical research publication as there is — compared the Nike Fuelband; Fitbit Zip, One, and Flex; the Jawbone UP 24; three iPhone apps, and a Galaxy 5 app. Participants each walked 500 and 1500 steps on a treadmill twice. A total of 560 observations were taken; each device or app got 27 or 28 data points.
(A “letter” to JAMA, by the way, is peer-reviewed but not as long, in-depth, or technical as a full journal article. It’s still pretty prestigious.)
The full letter is behind a paywall, but the first page has plenty of good data and conclusions. As a class, researchers found, the phones did best, averaging very close to the correct number of steps and varying by about 10 percent up or down. The Nike Fuelband did terribly, consistently undercounting by an average of about 20 percent and with a variability that renders its data meaningless. The Fitbit Flex undercounted slightly, with a range pretty close to what the phones reported.
But the real champs were the Fitbit One and Zip — devices that clip to your clothing, not hang on your wrist. In every test, they counted correctly and with only a tiny variance.
The implication here is interesting. Stuff you wear on your wrist would appear to be prone to the larges error, possibly because swinging your arm is not the same thing as a taking a step. Stuff you carry in your pocket is next best, maybe for the same reason: it doesn’t move around as much. Stuff that’s more or less stationary — clipped to clothing — is the most accurate of all. It’s not clear what happens when you combine two locations, like wrist and pocket, the way Samsung does and Apple will soon have.
There are a lot of reasons to wear devices on your wrist, fashion and design not least among them. If you want accuracy in a fitness device, though, the JAMA data implies that the wrist might be the wrong place for it — a finding that may have serious implications for the smartband business. For sure, though: I’m taking off my Fuelband, because it’s apparently spouting nonsense.