Limits

Layer is limited by certain constraints of the underlying components we ship with. We will lift these as demand increases.

• Single-node Aerospike. We enforce this for simplicity and also believe that a single large NVMe drive offers enough storage for almost every dataset.
• ~4,090 Turbopuffer namespaces. We use Aerospike sets for logical separation of data, which are limited by the Aerospike Community Edition AGPL license.
• ~3 TB cache size. Another limitation of the Aerospike license.

Not explicitly limited

These have no enforced ceiling, but practical limits exist and will surface under load.

• CRD instances (Index, Function, Pipeline, Scaling) — bounded only by the etcd and operator throughput of your Kubernetes cluster.
• Snapshot history per namespace — durable in S3, accumulates indefinitely; bounded by object storage cost.
• Search history retention — accumulates indefinitely in S3; no automatic expiry.
• Clickstream event volume — accumulates indefinitely in S3; no automatic expiry.
• UDF concurrency per function — KEDA scales replicas to match queue depth, bounded by your cluster’s capacity.
• Pipeline queue depth — segmented bulk queues store IDs in S3 manifests and keep segment state in Postgres; chunked document queues still use per-document Postgres rows.
• Document size and attribute count — bounded by Turbopuffer and Aerospike record limits, not by Layer.