Quiver on the edge: consistent scalable edge services

Hosting dynamic web services through proxies placed at the edge of the Internet is an upcoming trend that has the potential to scale these services to a very large number of geographically distributed clients. However; providing consistent access to shared mutable objects that make up the service, tolerating misbehaving proxies, and handling proxy disconnections, while still achieving the scalability and performance expected from such an architecture is a significant challenge. This dissertation presents Quiver, a distributed object system that supports consistent (serializable or strictly serializable) operations on shared objects by service proxies in a wide-area setting, while reducing the client perceived latency. We also present extensions to Quiver that detect compromised proxies attempting to violate Quiver's consistency properties, allow proxies to disconnect and efficiently reconnect without delaying operations from connected proxies, and optimize object access times by restructuring Quiver's communication network according to the workload. Quiver proxies are arranged in a "location-aware" rooted tree. In order to perform an update or a multi-object operation involving certain service objects, a Quiver proxy migrates those objects through the tree, to itself. Object migrations for operation processing ensures a serial execution of update and multi-object operations involving the same objects, and enables Quiver to achieve the desired consistency semantics, while optimizing for the (typically, more frequent) single-object read operations. Furthermore, when operations involving an object exhibit geographic locality---e.g., during business hours on one continent (and non-business hours on others)---the performance of these operations benefits from the object having been migrated to a nearby proxy. Other workloads benefit from Quiver dispersing the compute load across the proxies performing operations, and saving the costs of transmitting operation parameters over the WAN when these are large. Guaranteed consistency semantics for shared objects via migrations through a tree of proxies requires that the proxies are well-behaved, i.e., follow the protocol specifications. We, therefore, present an extension to Quiver that relaxes this assumption by allowing honest proxies to efficiently detect a misbehaving proxy that attempts to compromise the consistency of object accesses. Finally, this dissertation also discusses extensions to manage the rooted tree (an overlay) that connects the Quiver proxies, in order to improve the overall service availability and performance. In particular, we describe a distributed algorithm that constructs a fault-tolerant network on top of the tree, allowing proxies to efficiently reconnect to the primary partition (the partition containing the root), in case of proxy or link failures. This efficient reconnection algorithm reduces the "down-time" of Quiver proxies while avoiding some central point of reentry (e.g., the root) from being overloaded due to frequent reconnect requests. We also discuss extensions that heuristically restructure the tree to bring the proxies that frequently perform operations involving the same objects close to each other, guaranteeing an O (log n) (for n connected proxies) amortized object access cost for any workload. This dissertation details the protocols for implementing consistent object operations; for accommodating the dynamic addition, involuntary disconnection and voluntary departure of Quiver proxies; for detecting misbehaving proxies; for the construction of a fault-tolerant network over the tree; and for restructuring the tree according to the workload to reduce access costs. These algorithms are evaluated using a combination of simulations and experiments performed on PlanetLab and isolated local clusters.