Stephen Macke is an engineer at Meta working on notebook technologies. He completed his PhD at the University of Illinois, Urbana-Champaign in 2020, and has been obsessing about reactivity and state management in notebooks for way too long.
An underappreciated aspect of Jupyter and IPython experiences in general is their ability to autoreload Python modules during running sessions via the autoreload extension. At Meta, we began leveraging this functionality to power interactive test sessions that allow software engineers to quickly iterate on their projects without waiting for slow restart times.
However, the base autoreload algorithm suffers from a number of reliability issues and can easily crash, thereby necessitating a costly restart. In this presentation, we’ll describe references reload, which is our new and improved autoreload algorithm with a number of benefits over the basic autoreload functionality. We’ll show how we use references reload in production to save developers hours of time, and we’ll close with a concrete use case for development on top of Bento server, which is Meta’s internal version of the Jupyter notebook server.
In this talk, I'll introduce IPyflow, which is my stab at addressing the issue of statefulness in Jupyter notebooks. IPyflow is a Jupyter kernel designed from the ground up to annotate each symbol appearing in program text with metadata pertaining to how it relates to every other symbol.
This dependency graph is then leveraged to provide several interesting features, including:
- execution suggestions, for keeping the notebook state in sync with the code;
- dynamic backward slicing, to show the minimal code needed to reconstruct each symbol; and
- dynamic forward slicing, used to enable reactive execution.
Finally, I will also discuss IPyflow's reactive syntax extensions, which allow for opt-in reactivity at the granularity of individual symbols, as well as its interoperability with other libraries and frameworks such as ipywidgets and stickyland. I'll end the talk by showing how IPyflow can be used with the aforementioned projects to build fully reactive and interactive dashboards on top of JupyterLab.