Amit Rathi
Amit is a software developer & the founder of ReviewNB, a code review tool for Jupyter Notebooks.
He's a Carnegie Mellon graduate, previously worked at Amazon Music team in Seattle & helped build location stack at HyperTrack.
Amit is a strong proponent of automated testing & code reviews as they help us quickly build reliable software & learn from each other. He's deeply interested in the human side of software engineering- development processes, collaboration tools, and such.
Sessions
Key Takeaways
- How to build rich diffs & commenting for notebooks
- How to integrate notebook diff & commenting with version control platforms like GitHub & Bitbucket
- Propose a cross-compatible standard for storing comments/discussions inside notebook cell metadata
Summary
For the past 4+ years, I’ve built a notebook code review experience (ReviewNB) for GitHub as a solo bootstrapped developer. Thousands of organizations now use the service, including Apple, Airbnb, Lyft, Deloitte, Affirm, AWS, Meta Reality Labs, and NASA JPL.
This talk focuses on behind-the-scenes technical details such as,
- Challenges of building rich notebook diffs on top of GitHub / Bitbucket
- handling JSON diffs
- handling images, plots & other rich outputs
- Challenges of building discussion / commenting functionality for notebooks
- where & how to store notebook comments
- how to handle comments when underlying notebook changes
We'll briefly touch upon other tools that offer notebook diff & commenting (nbdime, Google Colab, Deepnote, Noteable).
Finally, we propose a cross-compatible standard for storing comments inside notebook cell metadata. If the Jupyter project endorses this, notebook comments, similar to the notebook format, would be compatible across platforms.
Summary
Today, Jupyter Notebooks are mostly confined to science, research & education. But notebooks can provide organizations with a powerful general-purpose “executable documentation” platform. A solid use case for this is DevOps & more specifically, IT incident response.
Technology teams usually have an on-call rotation with static wiki-style documentation to guide the on-call engineer. Jupyter Notebooks can replace static documentation with executable notebooks. E.g. “fetch service logs” and “rollback last deployment” can simply mean executing a code cell that’s available alongside the markdown instructions.
What are the benefits of executable vs. static documentation -
- Quick e.g. “check DB latency” is 1-click notebook code cell execution to plot latency graph vs. going to a third-party UI in the middle of an incidence
- Precise e.g. “promote read replica to master” can mean a series of steps & possibility of human error; codifying the steps in advance removes ambiguity & results in precise action.
“Executable documentation” is a simple yet powerful concept that can extend to other use cases such as - API documentation, developer onboarding, data visualization & reporting, scheduling routine tasks & so on. Think of it as executable GoogleDocs powered by Jupyter!
In this talk, we’d like to,
- Introduce the concept of an “executable documentation” platform powered by Jupyter
- Show a demo of how it’d work for IT incident response (https://www.youtube.com/watch?v=vvLXSAHCGF8)
- Talk about important considerations of building an executable documentation platform
- Illustrate other use cases for executable documentation