Running GPU Jupyter Workspaces on... - Andrii Ieroshenko, AWS & Gaurav Bhanderi, Amazon

Running GPU Jupyter Workspaces on Kubernetes: Scheduling, Autoscaling, and Policy Lessons - Andrii Ieroshenko, AWS & Gaurav Bhanderi, Amazon Kubernetes autoscaling is built for workloads that can be killed and rescheduled at any moment. A notebook is the opposite: its state lives in a running kernel, an eviction interrupts a person mid-work, and its GPU cannot be shared. We added GPU workspaces to jupyter-k8s, an open-source Kubernetes operator for Jupyter, and most of the hard questions turned out to be one question: which layer should own this knob, cloud instances, autoscaler, operator, infrastructure-as-code, or UI. The talk walks through the answers as lessons: keeping CPU pods off GPU nodes (and cheap GPUs off expensive ones) with a label and a taint per pool; why consolidation that evicts running pods does not work for notebooks, and what refusing it costs in bin-packing; idleness as two layers that have to compose, workspace idle-shutdown and node scale-to-zero; templates as the unit of per-pool policy (bounds, defaults, idle rules) and the API contract traps we hit; and the decisions that are hard to reverse later, MIG partitioning and dynamic resource allocation. The lessons apply to any Jupyter on Kubernetes, including JupyterHub. We close with where jupyter-k8s sits next to JupyterHub and the roadmap.