ST-PCT: Spatial-Temporal Point Cloud Transformer for Sensing Activity Based on mmWave

Kang, Liyu; Li, Zan; Zhao, Xiaohui; Zhao, Zhongliang; Braun, Torsten (2023). ST-PCT: Spatial-Temporal Point Cloud Transformer for Sensing Activity Based on mmWave. IEEE internet of things journal, 11(6), pp. 10979-10991. IEEE 10.1109/JIOT.2023.3329236

Text ST-PCT_Spatial-Temporal_Point_Cloud_Transformer_for_Sensing_Activity_Based_on_mmWave.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (3MB)

The millimeter-wave (mmWave) spectrum has become a core of wireless communication, which has the advantages of richer spectrum resources, larger communication bandwidth and smaller spectrum interference. Human Activity Recognition (HAR) by mmWave Radar based on point cloud attracts significant attention due to its nature of privacy-preserving, which is an important task of realizing Integrated Sensing and Communication (ISAC). This article proposes a framework of Spatial-Temporal Point Cloud Transformer (ST-PCT) to realize high precision of HAR, based on sequential point cloud after preprocessing from mmWave radar without voxelization. In ST-PCT, it consists of four enhanced components: (I) a framewise spatial neighbor embedding module to extract the local feature, (II) a temporal and spatial attention mechanism module to find connections within and across frames, (III) an optimized attention mechanism to improve the efficiency of feature extraction and (IV) a sensor fusion module with more motion information to improve the difference between activities. We experimentally evaluate the efficiency of our framework compared with several approaches based on the voxelization or point cloud directly. The experimental results have demonstrated that the proposed ST-PCT network greatly outperforms the other approaches in terms of overall Accuracy (oAcc), achieving 99.06% and 99.44% respectively on two datasets.

Interest & Impact

Downloads

2 since deposited on 22 Dec 2023
2 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item