ST-PCT: Spatial-Temporal Point Cloud Transformer for Sensing Activity Based on mmWave

Kang, Liyu; Li, Zan; Zhao, Xiaohui; Zhao, Zhongliang; Braun, Torsten (2023). ST-PCT: Spatial-Temporal Point Cloud Transformer for Sensing Activity Based on mmWave. IEEE internet of things journal, 11(6), pp. 10979-10991. IEEE 10.1109/JIOT.2023.3329236

[img] Text
ST-PCT_Spatial-Temporal_Point_Cloud_Transformer_for_Sensing_Activity_Based_on_mmWave.pdf - Published Version
Restricted to registered users only
Available under License Publisher holds Copyright.

Download (3MB)

The millimeter-wave (mmWave) spectrum has become a core of wireless communication, which has the advantages of richer spectrum resources, larger communication bandwidth and smaller spectrum interference. Human Activity Recognition (HAR) by mmWave Radar based on point cloud attracts significant attention due to its nature of privacy-preserving, which is an important task of realizing Integrated Sensing and Communication (ISAC). This article proposes a framework of Spatial-Temporal Point Cloud Transformer (ST-PCT) to realize high precision of HAR, based on sequential point cloud after preprocessing from mmWave radar without voxelization. In ST-PCT, it consists of four enhanced components: (I) a framewise spatial neighbor embedding module to extract the local feature, (II) a temporal and spatial attention mechanism module to find connections within and across frames, (III) an optimized attention mechanism to improve the efficiency of feature extraction and (IV) a sensor fusion module with more motion information to improve the difference between activities. We experimentally evaluate the efficiency of our framework compared with several approaches based on the voxelization or point cloud directly. The experimental results have demonstrated that the proposed ST-PCT network greatly outperforms the other approaches in terms of overall Accuracy (oAcc), achieving 99.06% and 99.44% respectively on two datasets.

Item Type:

Journal Article (Original Article)

Division/Institute:

08 Faculty of Science > Institute of Computer Science (INF) > Communication and Distributed Systems (CDS)
08 Faculty of Science > Institute of Computer Science (INF)

UniBE Contributor:

Braun, Torsten

Subjects:

000 Computer science, knowledge & systems
500 Science > 510 Mathematics
500 Science

ISSN:

2327-4662

Publisher:

IEEE

Funders:

Organisations 10 not found.; Organisations 10 not found.

Language:

English

Submitter:

Dimitrios Xenakis

Date Deposited:

22 Dec 2023 19:36

Last Modified:

13 Mar 2024 15:06

Publisher DOI:

10.1109/JIOT.2023.3329236

Uncontrolled Keywords:

Human Activity Recognition; Millimeter-wave Radar; Point Cloud; Transformer

BORIS DOI:

10.48350/190698

URI:

https://boris.unibe.ch/id/eprint/190698

Actions (login required)

Edit item Edit item
Provide Feedback