Jason Weill is a Senior Front-End Engineer at Amazon Web Services; a contributor to JupyterLab, Jupyter Governance, and Jupyter Scheduler; and a member of the JupyterLab Council, the Jupyter Security Working Group, and the Jupyter Diversity, Equity, and Inclusion Standing Committee. Public statements by Jason represent his opinions alone and not necessarily those of Amazon or any of its subsidiaries.
Generative artificial intelligence (AI) models are trained to generate new, previously unseen data (text, images, etc.). The generated data is both similar to the training data and a response to a user provided natural language prompt that describes a task or question. Recent generative AI models such as Amazon CodeWhisperer, Codex, Stable Diffusion, and ChatGPT have demonstrated solid results in performing a wide range of tasks involving natural language (content generation, summarization, question answering), images (generation, explanation, inpainting, outpainting, transformation), data (synthetic data generation, transformation, querying, etc.), and code (autocompletion, explanation, debugging, code reviews, etc.). In this talk, we describe open source work in Jupyter to enable end users to perform a wide range of development tasks using generative AI models in JupyterLab and the Jupyter Notebook.
Since their creation, Jupyter and IPython have always had an architecture and user interface for code generation through autocompletion. Indeed Jupyter users have been well trained to hit “tab” as they are coding to get contextual autocompletions for functions, variable, methods, properties, classes, etc. This architecture has already been used to integrate AI provided autocompletions into JupyterLab and the Jupyter Notebook (Kite, Tabnine). As generative AI models expand to perform other tasks, a generalization of the Jupyter autocompletion architecture is needed to enable users to perform potentially any task in Jupyter’s applications through a generative AI. In this talk we will describe an architecture and user experience for generative AI tasks in Jupyter.
The Jupyter generative AI architecture is founded on an extensible Jupyter Server API for registering generative AI models and the tasks they perform (represented as task prompts) with Jupyter. This enables third parties to quickly integrate their generative AI models into Jupyter and for end users to enable the models and tasks using the JupyterLab extension manager UI or a simple pip/conda install. Once models have been enabled, users working in JupyterLab or the Jupyter Notebook can 1) select anything (text, notebook cell, image, file, etc), 2) type a prompt to perform a task with the selection, and 3) insert the AI generated response in a chosen location (replace, insert below, new file, etc.). The task system is integrated with Jupyter’s MIME type output system, which enables GAI tasks to work with inputs and outputs of any MIME type. From the end users perspective, this enables them to enable and use models to perform tasks on text, code, images, data, audio, video, etc. This architecture also has APIs in Jupyter Server and JupyterLab that enable third parties to extend and integrate with the models and UI.
In the talk we will also give concrete examples of the AI driven tasks that can now be performed in JupyterLab. Examples include code refactoring, debugging, code transformation (e.g., Python to C/C++), code explanation (“generate and insert a markdown cell that describes this code”), synthetic data generation, data transformation, API documentation generation, technical content generation, and more. The focus of these examples will be less on the capabilities of the underlying AI models, and more on the seamless integration of the models into Jupyter to perform various tasks. Our assumption is that generation AI models will continue to improve and that Jupyter users will want a simple user experience that works with any such model.
Another potential application of this architecture is as a general purpose “playground” for exploring the capabilities of generative AI models. While not the primary goal of this work, the architecture effectively turns JupyterLab into a general-purpose generative AI playground. This enables users to quickly integrate models, and then use the familiar interface of Jupyter, with its support for editing files/notebooks, version control, real-time collaboration, etc., to play with generative AI. We discuss the pros and cons of this user experience and pose questions for future research and development.
This talk will be useful and of interest to 1) researchers who are building generative AI models to perform different tasks, 2) JupyterLab extension authors who would like to leverage generative AI, and 3) end users of Jupyter, who would like explore generative AI models and use them to perform tasks in their work.
Many Jupyter users want to run their notebooks in batch processing and production environments. Some popular and well-regarded open source projects and products now let users run and schedule notebooks as jobs. Until now, though, Jupyter itself has not offered its own notebook job capability. This talk will introduce you to Jupyter Scheduler, a new open source JupyterLab and Jupyter Server extension that lets all Jupyter users run and schedule notebook jobs via a simple yet powerful UI and API.
Jupyter Scheduler provides several benefits. First, notebook jobs can let users run more notebooks than their system could normally handle, without the overhead of setting up additional infrastructure. Second, notebook jobs improve the user experience of managing and parametrizing multiple long-running notebooks. Third, notebook jobs, unlike other types of production options, can include rich output (charts, data, widgets, etc.) and narrative text (markdown cells) in a periodic fashion, suitable for reporting. Fourth, in many cases, notebook jobs are a better way of moving to production than the traditional process of converting notebooks to Python scripts, containers, and CI/CD infrastructure.
Jupyter Scheduler, which is now part of the official Jupyter Server Subproject, provides an extensible notebook job architecture and user experience for notebook jobs. Not only does this capability enable users to run and schedule notebook jobs out of the box with Jupyter, it also enables third parties to plug different backends into the system to run notebooks as jobs on different platforms. In this talk, we will cover Jupyter Scheduler for end users who would like to run notebook jobs and for developers who would like to implement a new backend.
For end users, we will show how to install the extension, and use it to create, update, and manage both “run now” and scheduled jobs. We will demonstrate how to create and manage jobs with Jupyter notebooks, and walk through the different types of jobs supported by the scheduler. While this is not a full tutorial, you will be able to follow along and create your own jobs by installing the Scheduler extension prior to the talk.
For developers, we will also describe the architecture of this extension and how it allows you to customize both the user interface and the backend. Jupyter users are on a wide range of platforms: cloud providers, Kubernetes clusters, supercomputers, etc. Jupyter Scheduler addresses users’ needs by offering public extension APIs for running jobs on different platforms and for customizing the user interface for job creation. For example, Amazon Sagemaker Studio and Amazon SageMaker Studio Lab have created extensions for Jupyter Scheduler that let users run, schedule, and store notebook jobs on AWS through SageMaker.
Lastly, we will go over the future roadmap for this project, and talk about some of the interesting features expected in future releases. By the end of this session, you will have a good understanding of how to install, use and customize the Jupyter Scheduler extension in your notebook workflows. We are also very interested in learning about your unique use cases, so that we can adapt this extension to serve a wider Jupyter community.