Storage-Compute Separation
An architectural pattern in which the data persistence layer and the compute layer are deployed independently and scaled independently. It can improve elasticity and resource utilization, but introduces complexity in metadata coordination and consistency.
Core Facts
Timeline (last 90 days)
Decoupled storage-and-compute architectures are especially suited to scenarios in analytical workloads where compute demand fluctuates greatly, data volume keeps growing but access frequency is uneven.
Storage-compute separation means deploying the data persistence layer and the compute layer independently and scaling them independently; in a search system, index building, merging, and query serving can run on different node pools while sharing the same S3 data.
Storage-compute separation is not a universal answer but an explicit trade-off: trading latency and complexity for cost, elasticity, and durability.
Decoupling storage from compute allows teams to independently select a cheaper storage tier for cold data without being forced to upgrade expensive compute instances when expanding storage.
Traditional databases bind storage and compute on the same node, requiring both to be upgraded simultaneously when scaling.
Storage-compute separation architecture achieves second-level elastic scaling of compute resources by decoupling the storage layer from the compute layer, allowing storage to scale independently without data migration.
In a storage-compute separation architecture, data access requires crossing network boundaries, and even within the same availability zone, round-trip network latency is one to two orders of magnitude higher than that of local NVMe SSDs.
Storage-compute separation architectures are highly dependent on caching, and insufficient cache warm-up during service restarts, scale-out, or sudden changes in access patterns can cause latency spikes in the short term.
All Facts (8)
Storage-compute separation architecture achieves second-level elastic scaling of compute resources by decoupling the storage layer from the compute layer, allowing storage to scale independently without data migration.
75%UnverifiedDecoupled storage-and-compute architectures are especially suited to scenarios in analytical workloads where compute demand fluctuates greatly, data volume keeps growing but access frequency is uneven.
50%UnverifiedStorage-compute separation is not a universal answer but an explicit trade-off: trading latency and complexity for cost, elasticity, and durability.
50%UnverifiedStorage-compute separation means deploying the data persistence layer and the compute layer independently and scaling them independently; in a search system, index building, merging, and query serving can run on different node pools while sharing the same S3 data.
50%UnverifiedDecoupling storage from compute allows teams to independently select a cheaper storage tier for cold data without being forced to upgrade expensive compute instances when expanding storage.
50%UnverifiedTraditional databases bind storage and compute on the same node, requiring both to be upgraded simultaneously when scaling.
50%UnverifiedIn a storage-compute separation architecture, data access requires crossing network boundaries, and even within the same availability zone, round-trip network latency is one to two orders of magnitude higher than that of local NVMe SSDs.
50%UnverifiedStorage-compute separation architectures are highly dependent on caching, and insufficient cache warm-up during service restarts, scale-out, or sudden changes in access patterns can cause latency spikes in the short term.
50%