Vision-based Force Feedback Estimation for Robot-assisted Surgery using Instrument-constrained Biome

Our paper, "Vision-based Force Feedback Estimation for Robot-assisted Surgery using Instrument-constrained Biomechanical 3D Maps" has been accepted to IEEE Robotics and Automation Letters and ICRA 2018. We present a method for estimating visual and haptic force feedback on robotic surgical systems that do not feature significant force feedback to the operator.

This is done using a biomechanical model which is built on-the-fly with 3D reconstruction and meshing techniques from endoscopic images of the organ shape. Our approach allows for force feedback to be determined without the use of additional sensors, and only on endoscopic images acquired by stereoscopic camera. Contact forces can be estimated at the tip of the surgical tool by considering the tool as boundary conditions acting on the surface of the model. These contact forces also generate stresses that allows us to compose a new endoscopic image with visual force feedback for the surgeon.