Login Sign up

Jupyter for interactive in-situ visualization with ParaView/Catalyst

Alice Grosch, Christian Witzler

Audience level:
Intermediate

Brief Summary

When starting an interactive, scientific visualization with ParaView, usually the ParaView desktop application needs to be started. This takes the user out of the Jupyter environment. We have developed pvlink, which uses ParaViewWeb to bring the visualization directly into the notebook. In combination with ParaView Catalyst, in-situ visualization can happen right in the notebook.

Outline

ParaView is often used for scientific visualization and can be run on supercomputers for extremely large datasets as well as the local laptop for smaller data. Starting an interactive 3D Visualization in ParaView however can take some effort. For example, a user might have to start a VNC server on the supercomputing system first, tunnel the VNC traffic and then connect via a VNC viewer. Then the ParaView desktop application still has to be started on the remote desktop. Not only is this process lengthy, but it also requires some advanced user knowledge. But even just starting the desktop application on a local system requires the user to tab out of the Jupyter Notebook at the very least.

To bypass these problems and make it possible to keep the workflow going in a single Notebook, we have developed pvlink, a Jupyter widgets based notebook extension which uses ParaViewWeb to bring the visualization directly into the notebook.

On the backend side, a ParaViewWeb Server is started using Python. The widget on the frontend uses the ParaViewWeb JavaScript library. After connecting the backend and frontend via a websocket connection, a rendering viewer can be displayed directly in the output area of a notebook cell.

The process of starting server and connecting both ends can be as manual or hidden as the user desires. The extension supports Jupyter Server Proxy, so that a webserver started on a remote system does not have to be tunneled. In addition, the websocket connection can be password protected.

Once a renderer is displayed, no GUI elements will be present. A user can at this point use the mouse only to zoom, rotate or pan the camera. Nevertheless, because the webserver is Python based, a ParaView visualization pipeline can be entirely recreated using python code in the notebook by using the paraview.simple module. Loading sources into different views and manipulating them can be done by executing notebook cells while the changes are reflected in the pvlink rendering widgets.

For processing large data, a connection to a ParaView Server (not to be confused with the ParaViewWeb Server above) can be established to run ParaView in client/server mode. For example, a pvserver process can be started on a high-performance visualization cluster, which takes over the heavy processing. Then the user can connect to that server from within the Notebook. ParaView Catalyst enabled simulations allow for in-situ visualizations. If a Catalyst enabled simulation was started, the user can also connect to that. After running the necessary ParaView pipeline and displaying a pvlink renderer, the in-situ visualization can be rendered directly in the output cell of the Notebook. In short, with pvlink, even extremely large datasets can be handled and visualized, interactively and in-situ, with ParaView all from within a single Notebook.