Practical Urban Localization for Mobile AR

Emerging mobile apps render AR effects based on the places of interest (POI) that a user is currently in. To obtain the needed POI labels and a smartphone's camera position and orientation, such apps demand inexpensive localization. Yet, existing localization solutions either work poorly in urban areas or require expensive data collection. To this end, we advocate for an inexpensive, practical localization pipeline by integrating commodity vision operators. To instantiate the pipeline, we propose a system with three key designs: the cloud indexes image features as a forest rather than a monolithic tree; smartphones incrementally prefetch image features for on-device matching rather than uploading features to the cloud; smartphones tune the camera positioning algorithm dynamically based on its physical environment. Our preliminary results show that these designs can reduce the cost of image data collection by up to three orders of magnitude, reduce user-perceived delays, and scale to diverse AR resource demands and environments.