|
|
|
Camera Calibration and Stereo-based Reconstruction
Dr. Danail Stoyanov
Imperial College London |
Calibrating the intrinsic and extrinsic parameters of monocular and stereo laparoscopic imaging devices is an important step for optical 3D surface reconstruction. With this information reconstructing metric 3D shape from stereo images becomes feasible and with the emergence of stereoscopic devices for 3D visualisation of the surgical site, computational stereo has become a practical approach for 3D surface reconstruction in MIS.
In this tutorial, we will discuss the basic principles and current challenges in surface reconstruction from stereo images and we will demonstrate existing state-of-the-art tools in the field. We will then focus on the problem of creating robust algorithms that can be applied to the MIS environment and the practical difficulties of implementing real-time systems suitable for in vivo applications.
|
 |
Fast 3D Reconstruction Using Structured Light Methods
Prof. Dr. Marcos A. Rodrigues
Sheffield Hallam University |
In this tutorial we discuss the use of structured light scanners for the general problem of 3D surface reconstruction. We show that projecting patterns of light provide an inexpensive means of consistent 3D scanning at high resolution, in real-time and from single images. The main problem of such techniques is pattern decoding or stripe indexing, which can be substantially non-trivial and difficult to overcome in a reliable way. We discuss existing techniques and show how a minimal light coding in the projected stripes can resolve inherent ambiguities found in stripe patterns across surface discontinuities. We also discuss how our real-time solution using structured near-infrared light can overcome ambient illumination and used in a variety of medical contexts. |
 |
Foundations of Time-of-Flight Cameras and their Application to Surface Reconstruction
Prof. Dr. Andreas Kolb
University of Siegen |
Accurate and fast reconstruction of 3D scene objects is a challenging and long studied task in Computer Vision. Beside application areas like object detection, collision prevention, mixed reality, and gesture recognition, more and more medical applications integrate 3D sensing technologies.
The estimation of a range map is an essential prerequisite for 3D object reconstruction and analysis. Applying image analysis or laser scan techniques, range map acquisition is still a time-consuming and expensive part for each application system. Lower-priced, fast and robust alternatives for distance measurements are Time-of-Flight (ToF) cameras, which allow the instantaneous acquisition of full range maps with very little computational effort at video frame rates. Recently, significant advances have been made in producing these comparable low-cost and compact ToF-devices, which have the potential to revolutionize many fields in research and application.
This talk discusses the technical foundations of ToF-cameras, the characteristics of the acquired range data and sketches approaches to circumvent the most prominent measuring errors present in ToF-range data. Furthermore, the talk presents current examples of ToF-cameras in medical applications. |
Reconstruction of endoscopic ToF-data of a model of a human liver (image courtesy M. Winter, U. Erlangen). |
Deformable Shape-from-Motion
Prof. Adrien Bartoli
Université d’Auvergne |
Shape-from-Motion (SfM) solves 3D reconstruction using the camera’s displacement between multiple views as a visual cue. Over the past few decades, SfM has been solved under the assumption that the observed environment’s shape is rigid. Researchers have tried for about a decade to extend the theory and implementation of Rigid SfM to deformable environments under various shape deformation hypotheses and priors. We will present the state-of-the-art in Deformable SfM, based on the two-step paradigm of image registration and 3D reconstruction from registered images. The application of Deformable SfM to 3D reconstruction from laparoscopic video data will be presented and discussed along with preliminary experimental results. |
|
Temporal Feature Tracking for Robotic Assisted Endoscopic Surgery
Dr. Haytham Elhawary
Philips Research |
Visualization during minimally invasive bypass surgery on the beating heart can be enhanced by using a robotic guided endoscope and visual servoing from the endoscopic images. In order to achieve these objectives, accurate, robust and real-time motion tracking of features on the beating heart is required. Although there have been a number of tracking algorithms developed in literature, many of these solutions have found difficulty being translated into the clinic due to requirements such as endoscope calibration in the OR, attached markers to the heart or the need of non-standard surgical equipment such as high performance cameras. We propose the use of a simple marker-less approach combining optical flow tracking algorithms with a Speeded up Robust Features (SURF) based feature detector in order to track features which present good mathematical properties for tracking. Given the lack of ground truth in these applications, a series of metrics was developed in order to quantify accuracy, robustness and drift under a variety of circumstances. Experiments showing real-time feature tracking on a beating heart phantom with a standard monocular endoscope will be presented, followed by its application to uncalibrated visual servoing using a robotic guided endoscope. |
 |
Simultaneous Localisation And Mapping for Minimally Invasive Surgery
Dr. Peter Mountney
Siemens Corporate Research |
Image guidance for cardiac and abdominal Minimally Invasive Surgery (MIS) remains a challenging research area. Real time estimation of the laparoscopic camera's motion and generation of 3D tissue models are fundamental pre-requisites. The varied and changing MIS environment further complicates the problem. In this talk, vision based Simultaneous Localization And Mapping (SLAM) for MIS is introduced. The application of SLAM in MIS will be demonstrated for registration of multi modal intra operative registration, dynamic view expansion and Motion Compensated SLAM (MC-SLAM) for dynamic tissue modeling.
|
 |
Toolkits and Libraries
Dr. Peter Mountney, Dr. Danail Stoyanov, Dr. Lena Maier-Hein
|
The purpose of this talk is to provide an introduction to software libraries and toolkits for development of image-guided systems based on surface information. The participants will receive an overview of the available software solutions for processing video and range data, calibrating cameras, tracking image modalities/instruments, and registering data. In addition, several demos based on the open-source libraries/frameworks will be given. |
 |
Intra-operative Registration
Dr. Lena Maier-Hein
German Cancer Research Center |
Although laparoscopic interventions offer major benefits to the patients, including reduced post-surgical trauma and morbidity, as well as shorter hospitalization, they suffer from a limited field-of-view, loss of depth perception and a lack of tactile feedback. These issues can be compensated for by enhancing endoscopic video images with pre-operative planning data. The main challenge in this context is the intra-operative registration of the planning images with the patient's anatomy. This lecture will review state of the art approaches to intra-operative registration for augmented reality (AR) guidance in laparoscopic interventions. We will compare different methods, identify advantages and limitations and discuss practical challenges for clinical application.
|
 |
System Evaluation
Anja Groch
German Cancer Research Center |
Evaluation is one of the basic steps when developing a new technique. Based on the previously proposed methods in this tutorial, we show how to perform a reliable evaluation for different 3D surface reconstruction methods. This includes key issues like (1) the materials (phantom study vs. in-vitro/in-vivo study, texture properties of the surface, etc.) and setup, (2) the procedure to obtain ground truth surfaces and ground truth for the registration of reference surface and reconstructed surface, or (3) the comparison metrics (e.g. RMS surface distance, different surface descriptors, etc.).
We furthermore present an extensive study on a set of in-silico and in-vitro phantoms evaluating different surface reconstruction methods on the same data. |
 |
|
|
