Shifter: Containers for HPC

– Data-intensive (32-core Haswells, 128GB) partition, 1600 nodes – Compute-intensive (68-core KNLs, 90GB) partition – ~10k nodes, ~700k cores • Edison Cray XC30 … – 2.5PF – 357TB RAM – ~5000 nodes, ~130k cores • High speed parallel file system … – 28 PB scratch file system (Lustre) – 1.5 PB Burst Buffer (flash)* • High Speed Aires interconnect … *“Extreme I/O on HPC for HEP using the … Burst Buffer at NERSC” Wahid Bhimji et … – Breaks a lot of standard Linux work flows … – Runs SUSE, and doesn't upgrade often … – Sometimes unclear to users where is the best place to put their … • Many groups have turned to a new software package … – Supports Docker and other images (vmware, ext4 … – Intended for complex code stacks and codes with … – Users can create custom images in desired OS … – Upload image to docker hub and pull down on HPC … – Hooks to tie into the batch system