Is there latency overhead for Presto queries if everything fits into memory and doesn’t need to be distributed?
Presto is both in-memory and distributed, so each work has memory and uses it. However, Presto is not an in-memory database. Presto is the query engine and reads from storage underneath. This is where Presto’s caching capabilities become important.
If all the data required to satisfy a given query arrives from the connected data sources and fits within the memory of the Presto cluster, everything should work accordingly. However, Presto’s MPP, in-memory pipelining architecture will accelerate workloads further in two ways:
1) Presto will ensure data is divided across all the workers to bring maximum processing power and memory capacity to bear.
2) Presto’s generated query plan will ensure execution steps are distributed amongst all the workers such that processing takes place in parallel on the data across all workers simultaneously, as efficiently as possible. Furthermore, with its caching capabilities, Presto can accelerate query execution even further for specific workload patterns, further reducing latency.
If you want to get started with Presto, check out our docs to learn more about the Ahana Cloud managed service for Presto.