Using an autograding system with Jupyter Notebooks for independent student work requires the instructor to identify multiple possible solutions for a problem and still requires manual assignment of partial credit. This is often a challenge, particularly in disciplines like thermodynamics. In this talk, I will demonstrate a pair of Python packages to generate, distribute, and collect Notebooks.
Jupyter Notebooks are often used as the format for independent assignments for students, as homework or exam problems. In many disciplines, the use of autograding is convenient, and dramatically simplifies the task of grading for the instructor. However, autograding has three main disadvantages:
In my experience teaching thermodynamics for engineering students, identifying multiple solutions for a problem may be challenging. For many problems in thermodynamics, the behavior and solution procedure of the problem changes dramatically if the input conditions are changed. Thus, for the courses I teach, identifying multiple solutions is usually not feasible.
In addition, many non-computer-science engineering students are not very comfortable writing more complicated code, such as functions. The use of coding to solve problems is already a significant cognitive overhead, and if the additional requirement to write functions for the autograder was removed, that would be a benefit for many students.
Finally, assigning partial credit to problems is expected by many students, when their approach to a problem’s solution is appropriate, but they make a small error along the way and don’t get the blessed answer. This must be done manually, even in an autograding system.
The combination of these three factors has led me to pursue an alternate method of generating, distributing, and collecting assignments based on Jupyter Notebooks. This has taken the form of two open-source Python packages, called thermohw and convert_and_download, respectively.
The first package, thermohw, is a command-line interface that produces an assignment Notebook and a solution Notebook from a single input Notebook. The single input should contain the question, any required initial data, as well as the full solution for the problem. The input can use Jupyter Notebook extensions such as python-markdown to make it easier for the instructor to generate problems. The use of a single input is advantageous because it prevents multiple copies of the assignment from becoming out of sync.
When the input is processed by thermohw, it is converted into two Notebooks. Any values specified as python-markdown input are replaced with the actual value, and any images are embedded into the Notebook to ensure that the distributed Notebooks are complete. In one of the output Notebooks, the solution to the problem is stripped and replaced with placeholder cells prompting students for input. This Notebook is distributed to students as the assignment. The other output Notebook contains the complete solution to the problem and is distributed after the assignment is handed in.
After students complete the assignment, they use the convert_and_download package to convert their Notebooks into a single PDF. convert_and_download is an extension for the Jupyter Notebook that allows multiple Notebooks to be selected and downloaded as a single PDF file. It does this by collecting the bytes of the PDF generated by nbconvert and stitching them together with the pdfrw package.
Another advantage of convert_and_download is that it automatically extracts images from the students’ Notebooks and includes them in the PDF output. This allows students to embed images of their hand-written work on the problem, to be graded for partial credit. This also allows students who are not familiar with Markdown to provide an explanation of their process. Finally, in thermodynamics in particular, sketches of the process described in the problem are often required for a problem, which are uploaded as images to a cell.
The PDF that is generated by convert_and_download is uploaded to the university learning management system for grading by the instructor or teaching assistants. The grading experience is quite similar to grading assignments handed in on paper. Since students show all of their work and code in the Notebook, all of the required information is present in the single document.
This talk is pitched at an intermediate audience. Prior experience with using Jupyter Notebooks would be helpful for the audience to have, and if they have experience how Jupyter Notebooks are stored as files, they will benefit more from the later sections of the talk.
The key takeaways for the audience will be
convert_and_download: https://github.com/bryanwweber/convert_and_download
thermohw: https://github.com/bryanwweber/thermohw