AI for Remote Robotic Telesurgery


Robotic teleoperation brings great potential for advances within the field of surgery. The ability of a surgeon to reach patient remotely opens exciting opportunities. Early experience with telerobotic surgery has been interesting, but the clinical feasibility remains out of reach, largely due to the deleterious effects of communication delays. Teleoperation tasks are significantly impacted by unavoidable signal latency, which directly results in slower operations, less precision in movements, and increased human errors. Introducing significant changes to the surgical workflow, for example by introducing semi-automation or self-correction, present too significant a technological and ethical burden for commercial surgical robotic systems to adopt. We develop innovative solutions that reduce the risk of long distance operations while still giving the surgeon full control of the operation . 


One solution is to give the operator immediate visual feedback of the surgeons motions by overlaying an Augmented Reality (AR) rendering a prediction of the operators move on the delayed video feed. This is also known as a predictive display. To realize this solution active tracking of the surgical robotic tools is required because relying on kinematic calibrations to localize the surgical robotic tools fails due to the millimeter scale of a surgical operation and the often utilized cable driven actuation. Also an adaptive transparency filter is utilized to prevents confusion arising from overlapping virtual and real visuals of the robot arms. The results from a user study with both experts and novices showed with statistical significance that using the predictive display decreased time to complete task while having no effect on error rates when operating under delay.


Another solution developed simply changes the scaling relationship between the surgeons motions in the master console and the patient side manipulators which hold the surgical robotic tools. We call this the motion scaling solution. Motion scaling has been shown to be an effective method of improving accuracy for surgeons when operating under normal conditions. We extend this idea to the remote telerobotic surgery and show that the motion scaling solutions reduces errors while having a minimal effect on time to complete tasks.

Students and Collaborators


Florian Richter

Zihyun Chiu

Fei Liu

Ryan Orosco

Emily Funk


Compensatory motion scaling for time-delayed robotic surgery

R.K. Orosco, B. Lurie, T. Matsuzaki, E.K. Funk, V. Divi, F.C. Holsinger, S. Hong, F. Richter, N. Das, M. C. Yip

Surgical endoscopy, pp. 1-6 [springer]

Augmented Reality Predictive Displays to Help Mitigate the Effects of Delayed Telesurgery

F. Richter, Y.F. Zhang, Y.H. Zhi, R.K. Orosco, M.C. Yip

IEEE International Conference on Robotics and Automation - ICRA2019. May 20-24, 2019. Montreal, Canada. (Accepted). [pdf][vid]


Motion Scaling Solutions for Improved Performance in High Delay Surgical Teleoperation.

F. Richter, R.K. Orosco, M.C. Yip

IEEE International Conference on Robotics and Automation - ICRA2019. May 20-24, 2019. Montreal, Canada. (Accepted). [pdf][vid]

New adaptive scaling strategies lowers the barrier to telerobotic surgery in communication-delayed, far-forward environments

F. Richter, Z. Ye, W. Xu, A. Bamford, E. Wirtz, R. Orosco, S. Hong, M. Yip

Abstract, Military Health Systems Research Symposium, Kissimmee, FL, Aug. 20-23, 2018.

Telesurgical Robotic Operative Network:  Addressing signal latency in telesurgery with machine learning development using microsurgical platform

S. Hong, R. Orosco, M. Yip, S. Bevans, E. Wirtz, K. Groom, M. Tabata, J. Zhou, V. Divi, J. Robitschek, J. Sterbis, F. Holsinger.

Military Health Systems Research Symposium, Kissimmee, FL, Aug. 26-29, 2017.

Performance Analysis of a Haptic Telemanipulation Task under Time Delay 

M. Yip, M. Tavakoli and R. Howe

Advanced Robotics, vol. 25, no. 5, pp. 651-673, 2011. [pdf]

Performance Analysis of a Manipulation Task in Time-Delayed Teleoperation 

M. Yip, M. Tavakoli and R. Howe

Proceedings from IEEE/RSJ International Conference on Intelligent Robots and Systems - IROS 2010, pp. 5270-5275, 2010. [pdf]