We present “SlicerJupyter”, a 3D Slicer kernel allowing developers to implement image processing and visualization workflows in a notebook. 3D Slicer is C++ desktop app including tools for DICOM interop, GPU accelerated volume viz, non-linear transformations, device interfaces, and other features from core analysis to full-scale specializations such as radiation therapy/neuroimaging/astronomy.
3D Slicer (or Slicer for short) is a C++ desktop application that uses Qt, ITK, and VTK libraries for visualization and medical image analysis. 3D Slicer contains a wide range of tools to handle research and clinical workflows such as DICOM interoperability, GPU accelerated volume visualization, non-linear transformations, device interfaces, and many other features ranging from core analysis primitives to full-scale specializations for domains such as radiation therapy, neuroimaging, and even astronomy.
Slicer’s embedded Python interpreter makes all its features accessible with the Python programming language. Slicer has a simple built-in console to run Python commands interactively and can run Python scripts from files, but these are not as convenient as cell-based interactive notebooks, which have become popular among data scientists and researchers in recent years.
In this talk, we present “SlicerJupyter”, a 3D Slicer kernel allowing developers to implement complete data processing workflows in a notebook, using the powerful medical imaging and bioimaging tools in 3D Slicer. We will describe how Jupyter interactive widgets (sliders, buttons, etc.) can be used to control Slicer, modify data, or adjust processing and visualization parameters.
We will review the different levels of interactivity:
Level 1: Standard Jupyter widgets display application specific objects by automatic conversion of application-specific data objects to standard Python objects. For example, Slicer markup fiducial lists are displayed as a nicely formatted table and model nodes are rendered as 3D objects.
Level 2: Static image widgets display content that the desktop application renders. These widgets can be made interactive by modifying data and rendering parameters using additional standard widgets. This makes rich visualization capabilities — sophisticated rendering various data types, rendering of very large data sets, etc. directly available in Jupyter.
Level 3: Dynamic viewer widgets display 2D and 3D views rendered by the desktop application. Mouse and keyboard events are forwarded to the desktop application that allows zooming/rotating views, and to utilize all 3D interactions implemented in the desktop such as placing annotations, making measurements, or segmenting images the same way as if it was done on the desktop appication’s screen. This is implemented in the Slicer Jupyter kernel using ipycanvas and ipyevents packages.
Level 4: Full desktop graphical user interface integration. Users can see parts of the application window rendered in notebook cells, including standard desktop widgets (sliders, menus, etc.). It is implemented using noVNC and TigerVNC in Slicer Jupyter. This is particularly useful when the application runs on a remote server.