He will present the lecture ‘Generative AI-empowered 3D human pose tracking’
Abstract: In recent years, 3D human activity recognition and tracking has become an important topic in human-computer interaction. To preserve the privacy of users, there is considerable interest in techniques without using a video camera. In this talk, we first present RFID-Pose, a vision-assisted 3D human pose estimation system based on deep learning (DL), as well as its variations with enhanced generalizability to unseen test subjects and environments. The performance of DL models depends on the availability of sufficient high-quality radio frequency (RF) data, which is more difficult and expensive to collect than other types of data. To overcome this obstacle, in the second part of this talk, we present generative AI approaches to generate labeled synthetic RF data for multiple wireless sensing platforms, such as WiFi, RFID, and mmWave radar, including a conditional Recurrent Generative Adversarial Network (R-GAN) approach and diffusion/latent diffusion based approaches. Finally, we propose a novel framework that leverages latent diffusion transformers to synthesize high quality RF data. This synthetic data augments limited datasets, enabling the training of a subject-adaptive transformer-based kinematics predictor to estimate 3D poses with temporal smoothness from RFID data. Additionally, we introduce a latent diffusion transformer with cross-attention conditioning to accurately infer missing joints in skeletal poses, completing full 25-joint configurations from partial (i.e., 12-joint) inputs. This is the first method to detect 20+ distinct skeletal joints using Generative-AI for RF sensing-based continuous 3D human pose estimation (HPE).
Bio: Shiwen Mao is a Professor and Earle C. Williams Eminent Scholar and Director of the Wireless Engineering Research and Education Center at Auburn University. Dr. Mao’s research interest includes wireless networks, multimedia communications, RF sensing and IoT, smart health, and smart grid. He is the editor-in-chief of IEEE Transactions on Cognitive Communications and Networking, a member-at-large on the Board of Governors and the Technical Committee Board Director of IEEE Communications Society, and Vice President of Technical Activities of IEEE Council on Radio Frequency Identification (CRFID). He is a co-recipient of several technical and service awards from the IEEE. He is a Fellow of the IEEE.