Cornell's SciFi Lab has unveiled EchoMotion, a system that tracks a person's full-body pose using only the built-in speaker and microphone of an off-the-shelf smartwatch.

The watch's speaker emits inaudible ultrasonic signals that surround the wearer in a sphere of sound waves, and the microphones pick up echoes bouncing off limbs as they move. A lightweight AI model translates those echo patterns into the positions of 12 joints, from head to ankles.

The team presented the work at UbiComp/ISWC in Shanghai on October 13, and the study appears in the Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies.

This is the first time researchers can track whole-body movement with ultrasonic sound waves from a single commercial smartwatch, and the privacy angle is the hook: no camera needs to be set up at the gym, and acoustic signals generate far less data to process.

The prototype sensing drew about 12 milliwatts, roughly 40 times less than a wearable camera, and a 30-minute session on a Galaxy Watch 6 used only 5% to 6% of the battery.

Tech Xplore quotes lead author Saif Mahmud calling it 'a very important jump for body tracking,' noting that today's accelerometers and gyroscopes only sense the device itself.

After building a 3D-printed prototype with a mic and speaker pair on each side, the team tested EchoMotion on an unmodified Samsung Galaxy Watch 6 with 15 new volunteers.

For users the system had never seen, average joint-position error was under 4 inches, dropping to about 2.4 inches after a one-time 12-minute calibration.

Digital Trends reports that the same AI model, without any retraining, also ran on a Galaxy Watch 7 and a Google Pixel Watch 3 with only a slight dip in accuracy, and that the entire package is less than 30MB and deployable on Wear OS.

Digital Trends reports that an exercise demo on the Galaxy Watch 6 recognized 20 gym workouts, including squats and dumbbell lateral raises, with 95.4% accuracy, even with gym music blaring at around 76 dB.

The team now wants a personal fitness agent that counts reps and corrects form, and is exploring EchoMotion alongside smart glasses for XR body tracking.

The main caveat is people: accuracy dipped in a narrow hallway when echoes bounced off passersby, though furniture and road noise barely interfered. Mahmud expects more training data from thousands of users will make it robust even in crowded places like the subway.