Demo of Laparoscopic Liver Surgery System

Here, at UCL, we demonstrate the SmartLiver IGS system.

• The aim is to display CT models on top of live laparoscopic video

• Talk overview of system components

  1. Optical Tracker

  2. Endoscopic stack

  3. NVidia SDI input

• Perform “manual” registration using NiftySLS Liver Surgery plugin

• Perform “ICP” registration using NiftySLS Liver Surgery plugin

(all steps will probably require live demo/assistance).

• Discuss Coordinate Systems

• Compare “Laparoscopic System” with “Home Made” System (see Task 2)

For Laparoscopic System:

1. Laparoscope is tracked (T1)

2. Laparoscope has a camera, the intrinsic parameters (zoom, focus, centre of projection) are calibrated, and normally assumed to be fixed. However, it still has a position relative to the tracking balls on the Laparoscope. This relative transformation is called Hand-Eye (from robotic literature), here marked as T2.

3. When a surface is captured, we transform coordinates into “Tracker Space”, which is synonymous with “World Space”. To render things at the right position on screen, you simply need to mimic the transformations with your rendering system (VTK). In our case, the camera sits in “World Space”. So, once we register CT coordinates to “World Space”, here marked as T3, we can project the CT data, into the virtual camera, here marked as T4.

4. So, the basic rendering system is a case of geometry and setting up the rendering engine to mimic the correct geometry.

For “Homemade System”:

1. We use a webcam. So, the camera intrinsic parameters are calibrated, exactly the same as the laparoscope.

2. We attach a reference board, with a set of ArUco markers. This defines the world coordinates. One corner is defined to be the origin etc. So, in the diagram below T5 is equivalent to T2 then T1 in Fig 1.

3. So, if we register CT coordinates to the Reference, this is equivalent to registering the CT coordinates to the tracker in the example above. So, in both cases, these are labeled T3.

4. This registration is achieved by tracking a separate object (pointer), that can measure points relative to the Reference object? i.e. measures points in “World Space”.

5. So, this setup is slightly simpler than the laparoscope, but in principal, it’s enough to understand how such a system can be set up. Different systems might have slightly different arrangements of rigid body matrices, but the principal is the same.