Back To Schedule
Thursday, July 11 • 2:00pm - 2:20pm
From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

We present gg, a framework and a set of command-line tools that helps people execute everyday applications—e.g., software compilation, unit tests, video encoding, or object recognition—using thousands of parallel threads on a cloud-functions service to achieve near-interactive completion time. In the future, instead of running these tasks on a laptop, or keeping a warm cluster running in the cloud, users might push a button that spawns 10,000 parallel cloud functions to execute a large job in a few seconds from start. gg is designed to make this practical and easy.

With gg, applications express a job as a composition of lightweight OS containers that are individually transient (lifetimes of 1–60 seconds) and functional (each container is hermetically sealed and deterministic). gg takes care of instantiating these containers on cloud functions, loading dependencies, minimizing data movement, moving data between containers, and dealing with failure and stragglers.

We ported several latency-sensitive applications to run on gg and evaluated its performance. In the best case, a distributed compiler built on gg outperformed a conventional tool (icecc) by 2–5×, without requiring a warm cluster running continuously. In the worst case, gg was within 20% of the hand-tuned performance of an existing tool for video encoding (ExCamera).


Sadjad Fouladi

Stanford University

Francisco Romero

Stanford University

Dan Iter

Stanford University

Qian Li

Stanford University

Shuvo Chatterjee


Christos Kozyrakis

Stanford University

Matei Zaharia

Stanford University

Keith Winstein

Stanford University

Thursday July 11, 2019 2:00pm - 2:20pm PDT
USENIX ATC Track I: Grand Ballroom I–VI