Quality-focused Active Adversarial Policy for Safe Grasping in Human-Robot Interaction (QFAAP)

The QFAAP is designed to enhance the safety of vision-guided robot grasping in Human-Robot Interaction (HRI) scenarios. It introduces an Adversarial Quality Patch (AQP) and a Projected Quality Gradient Descent (PQGD) that adapts to human hand shapes from the perspective of benign adversarial attacks, which can be used to reduce the grasping priority of hands and nearby objects, enabling robots to focus on safer, more appropriate grasping targets.

Paper | Demo Videos

If you use this work, please cite:

@inproceedings{clee2025qfaap,
	title={Quality-Focused Active Adversarial Policy for Safe Grasping in Human-Robot Interaction},
	author={Chenghao, Li Razvan, Beuran and Nak Young, Chong},
	booktitle={IEEE Transactions on Automation Science and Engineering},
	year={2025}
}

Installation

This code was developed with Python 3.8 on Ubuntu 22.04. Python requirements can installed by:

pip install -r requirements.txt

Hardware

The code was deployed by the UFactory 850/Xarm5 Robot and Intel RealSense D435i.

1. UFactory Robot API: https://github.com/xArm-Developer/xArm-Python-SDK.
2. Intel Realsense API: https://github.com/IntelRealSense/librealsense.

Datasets

Currently, all datasets are supported.

Cornell Grasping Dataset

1. Download the and extract Cornell Dataset.

OCID Grasping Dataset

1. Download and extract the OCID Dataset.

Jacquard Grasping Dataset

1. Download and extract the Jacquard Dataset.

Pre-trained Grasping Models

All pre-trained grasping models for GG-CNN, GG-CNN2, GR-Convnet, and others can be downloaded from here.

Pre-trained AQP

All AQP trained by different grasping models and datasets can be downloaded from here.

Pre-trained Hand Segmentation Models

All pre-trained Hand Segmentation models can be downloaded from here.

Training/Evaluation

Training for AQP is done by the AQP_training.py. Training for Grasping model is done by the train_grasping_network.py. And the evaluation process is followed by the training.

Predicting

1. The offline and realtime prediction of QFAAP is done by the QFAAP_offline.py and QFAAP_realtime.py.
2. For the deployment of real-time hand segmentation, please refer to this repository https://github.com/Unibas3D/Upper-Limb-Segmentationp

Running on a Robot

Grasping with QFAAP is done by the QFAAP_grasping.py or QFAAP_dynamic_grasping.py.