Stage: Query Execution Time Prediction in Amazon Redshift
CoRR(2024)
摘要
Query performance (e.g., execution time) prediction is a critical component
of modern DBMSes. As a pioneering cloud data warehouse, Amazon Redshift relies
on an accurate execution time prediction for many downstream tasks, ranging
from high-level optimizations, such as automatically creating materialized
views, to low-level tasks on the critical path of query execution, such as
admission, scheduling, and execution resource control. Unfortunately, many
existing execution time prediction techniques, including those used in
Redshift, suffer from cold start issues, inaccurate estimation, and are not
robust against workload/data changes.
In this paper, we propose a novel hierarchical execution time predictor: the
Stage predictor. The Stage predictor is designed to leverage the unique
characteristics and challenges faced by Redshift. The Stage predictor consists
of three model states: an execution time cache, a lightweight local model
optimized for a specific DB instance with uncertainty measurement, and a
complex global model that is transferable across all instances in Redshift. We
design a systematic approach to use these models that best leverages optimality
(cache), instance-optimization (local model), and transferable knowledge about
Redshift (global model). Experimentally, we show that the Stage predictor makes
more accurate and robust predictions while maintaining a practical inference
latency and memory overhead. Overall, the Stage predictor can improve the
average query execution latency by 20% on these instances compared to the
prior query performance predictor in Redshift.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要