Researchers have developed a new method to generate realistic synthetic data for wearable motion sensors, aiming to improve how well computers can recognize human activities like walking or sitting. This is important because collecting and labeling real sensor data from many different people is costly and time-consuming, and existing models often struggle to work well on individuals they haven’t seen before. By creating high-quality synthetic data in a smart way, the new approach helps activity recognition systems learn better and generalize across diverse users, potentially making fitness trackers, health monitors, and other wearable devices more reliable.
Key Takeaways
- The team used a pretrained biosignal encoder called Bio-PM to transform raw inertial sensor data into a compact representation, then generated synthetic data within this space.
- Two types of generative models were tested: an autoregressive transformer and a diffusion model, both designed to produce sequences of tokens representing sensor signals.
- The diffusion model’s synthetic data was realistic enough to train activity classifiers that performed well on real-world data, achieving a macro-F1 score of 0.68 in a train-on-synthetic, test-on-real setup.
- Augmenting real training data with synthetic tokens improved classification accuracy across 51 subjects, especially for difficult-to-distinguish activities, raising the macro-F1 score from 0.729 to 0.809.
Human activity recognition (HAR) systems rely on inertial measurement units (IMUs), which are sensors embedded in smartphones and wearables that track motion through accelerometers and gyroscopes. These systems need lots of labeled data—segments of sensor readings tagged with the correct activity—to learn patterns and make accurate predictions. However, gathering such data from a wide variety of people can be expensive and labor-intensive. Moreover, models trained on one group often perform poorly when tested on new individuals due to differences in how people move.
To address these challenges, the researchers explored generating synthetic IMU data not by directly creating raw sensor signals, but by working in the “representation space” of a pretrained biosignal encoder called Bio-PM. This encoder transforms 10-second windows of sensor data into sequences of tokens—compact numerical embeddings that capture essential features of the motion. By keeping the encoder fixed (“frozen”), the team then trained two types of decoders to generate new token sequences conditioned on the activity class: an autoregressive transformer decoder that predicts tokens one after another, and a diffusion decoder that gradually denoises random token sequences to produce realistic ones.
Evaluating the synthetic data involved checking how closely its statistical properties matched real data and visualizing the embeddings to confirm realistic structure. Importantly, the researchers tested whether classifiers trained solely on synthetic data could correctly identify activities in real sensor readings—a method known as train-on-synthetic, test-on-real (TSTR). The diffusion decoder’s synthetic tokens enabled a classifier to achieve a macro-F1 score of 0.68, indicating fairly good recognition performance on unseen real data. Further, when synthetic data was combined with real data, classification accuracy improved significantly, especially for activities that are dynamically similar and harder to distinguish.
Additionally, the team developed a PatchDecoder to approximately reconstruct raw sensor waveforms from the generated tokens, demonstrating the potential to recover interpretable signals from the synthetic embeddings. This step could help bridge the gap between abstract token representations and actual sensor outputs.
These findings suggest that generating synthetic IMU data within a pretrained encoder’s representation space is a promising strategy for augmenting training datasets in human activity recognition. By improving cross-subject generalization, this approach could enhance the robustness of wearable health and fitness applications without the need for costly data collection from many individuals. Future work may focus on refining the generation models for even higher fidelity and exploring other types of biosignals to extend the benefits of synthetic data augmentation.
Based on research published on arXiv by Deepali Singh, Deepesh Parpani, Aviral Gupta.
