SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical Assistants

Moghani, Masoud; Doorenbos, Lars; Panitch, William Chung-Ho; Huver, Sean; Azizian, Mahdi; Goldberg, Ken; Garg, Animesh (October 2024). SuFIA: Language-Guided Augmented Dexterity for Robotic Surgical Assistants. In: International Conference on Intelligent Robots and Systems.

Text 2405.05226v1.pdf - Published Version Restricted to registered users only Available under License Publisher holds Copyright. Download (2MB) | Request a copy

In this work, we present SuFIA, the first framework for natural language-guided augmented dexterity for robotic surgical assistants. SuFIA incorporates the strong reasoning capabilities of large language models (LLMs) with perception modules to implement high-level planning and low-level control of a robot for surgical sub-task execution. This enables a learning-free approach to surgical augmented dexterity without any in-context examples or motion primitives. SuFIA uses a human-in-the-loop paradigm by restoring control to the surgeon in the case of insufficient information, mitigating unexpected errors for mission-critical tasks. We evaluate SuFIA on four surgical sub-tasks in a simulation environment and two sub-tasks on a physical surgical robotic platform in the lab, demonstrating its ability to perform common surgical sub-tasks through supervised autonomous operation under challenging physical and workspace conditions. Project website: orbit-surgical.github.io/sufia

Interest & Impact

Downloads

0 since deposited on 16 Jul 2024
0 in the past 12 months

Citations

5 Citations in Web of Science ®
6 Citations in Scopus

Search

in Google Scholar™

Services

Actions (login required)

Edit item

Actions (login required)

Edit item