Authors: Michael Wawrzoniak, Rodrigo Bruno, Ana Klimovic, and Gustavo Alonso
This paper, proposes the MetaQ, a proof of concept design of the EPQE paradigm. The goal is to to run unmodified data processing engines on serverless FaaS platforms and show that the services can be instantiated on demand when queries arrive.
Motivation
Long-running query engines have several limitations:
- They incur costs even if they are idle.
- Most distributed query engines lack elasticity, which leads to over-provisioned deployments.
- As workload diversity increases, each query might benefit from a different configuration and/or engine deployment (e.g., involving accelerators, caches, parallelism level, etc.), resulting in queries running in a less than optimal setting.