JupyterCon 2023

Audience level: Intermediate
- Project Jupyter users, looking to build Jupyter Extensions
- Developers hoping to deep dive into Project Jupyter

TLDR;

In this session, we will introduce 2 challenges we faced on Project Jupyter while developing our extension - MakinaRocks Link; and how we solved these challenges and how we implemented our ideas

• Handling how to display cell outputs
• Handling how to display error messages

We hope that sharing our lessons learnt from our deep dive into Project Jupyter can help other software developers of similar Project Jupyter-related extensions.

Background & Introduction

MakinaRocks Link

MakinaRocks Link is a JupyterLab extension that allows you to create pipelines on Jupyter by converting cells into components and setting parent-child relationships between components. We have built this extension to allow users access to all features originally available in the Jupyter environment. They say a picture is worth a thousand words - click below to find out more about how to create pipelines and run these pipelines.

• create-pipeline
• run-pipeline

2 challenges we wanted to solve

• 1 Displaying only the outputs of a child cell, when multiple parent cells are executed internally
  • When we run a certain component in a pipeline, we execute all dependent components as well. Then, what would be the ideal output when that child component is run?
  • Before we solved the issue - The outputs of all parent components are also displayed
    • unresolved-case-handling-outputs-gif
• 2 Displaying error messages like Jupyter
  • We hoped to replicate the useability of JupyterLab as much as possible. So, when an error occurs, we wished to replicate the JupyterLab error message and not expose our source code that is operating in the background, acting as unnecessary noise to the user.
  • Before we solved the issue - When an error occurs, the extension source code is also displayed, making it difficult for users to find the error messages about their own source code.
    • unresolved-case-handling-error-message-gif

#1. Displaying only the outputs of a child cell, when multiple parent cells are executed internally

• Our goal was to display only the output from the “executed component” and store the other outputs from other components internally
  • resolved-case-handling-outputs-gif
• Description of the original logic for displaying the output of a child cell with multiple parent cells on JupyterLab
• Development steps to display only the output from the “executed child component”
  • Inspired by capture_output class from IPython.utils module
  • Learnt that stdout & stderr are related to OutStream object, and the displays(images etc.) are related to DisplayPublisher object.
  • Our solution - snitched Outstream and DisplayPublisher objects and modified the write and publish methods respectively to implement our goal.

#2. Displaying error messages like Jupyter

• Our goal was to provide the identical Jupyter experience that users are familiar with. (We wanted to minimize time lost by having to learn something new. Hence, displaying error messages in the following way allows users to experience the same Jupyter error message environment.
  • resolved-case-handling-error-message-gif
• Description of the original logic of displaying error messages on JupyterLab
• Development steps for displaying the error message in the ‘right’ way
  • Found showtraceback method, which is used on ipykernel.zmqshell module, when ZMQInteractiveShell object displays error messages.
  • Eureka! Found _render_traceback_() method within showtraceback method, that allows customizations.
  • Our solution - wrote an algorithm to implement _render_traceback_() to display only the actual error message with zero noise.

Closing Remarks

We have shared our case of deep diving into Project Jupyter, and how we solved the challenges of showing the ‘right’ cell outputs and the ‘right’ error messages. We hope that Project Jupyter users can enhance their understanding of Jupyter and be inspired to solve more challenges with our case story.