Unleashing JupyterHub: Exploiting Resources Without Inbound Network Connectivity Using HTCondor

In recent years Jupyter notebooks have conquered class rooms and some scientists also enjoy their convenience to quickly evaluate ideas and check whether a more detailed study is justified. To lower the threshold for getting started with Jupyter notebooks and to ease sharing and collaborative use, offering a JupyterHub service is tempting. However, offering such a service for a larger science class also requires a compute backend with sufficient resources such that hundreds of notebooks can be run simultaneously. Since resource usage for teaching activities typically fluctuates significantly over the year, dedicated compute resources seem inefficient. In this paper we present an alternative by exploiting an existing high throughput computing cluster (BAF2) at the University of Bonn, which comes with the additional advantage that scientific users may use the very same software and data environment they also select for their batch jobs. To implement this, we used a novel approach which allowed us to integrate BAF2 execute nodes although they do not have inbound network connectivity. Therefore, it does not touch the security concept of the cluster. The very same technique can be used to integrate any compute resources without inbound network connectivity and thus allows one to overcome usual firewall restrictions. This design also simplifies exploiting remote resources e.g. offered by resource federations or cloud providers.