API

Search

POST /v2/namespaces/{namespace}/search takes a query string and returns ranked, scored rows. Text in, rows out — no vector to compute, no legs to assemble, no fusion math to write.

/query is unchanged and stays the turbopuffer-shaped wire. /search is Layer-native.

The pipeline

Every request walks the same seven stages. Two of them call a model, and both models are yours to choose — see Models and keys.

POST /v2/namespaces/{namespace}/search
     { "query": "...", "top_k": 10 }
                 │
                 ▼
   ┌───────────────────────────┐
   │ 1  embed                  │ ──▶ [ query model ]
   └───────────────────────────┘     the schema's embed: attribute
                 │ vector
                 ▼
   ┌───────────────────────────────────────────────┐
   │ 2  expand legs                 16-leg budget  │
   │    ann · bm25 per attribute · fuzzy per token │
   └───────────────────────────────────────────────┘
                 │
                 ▼   one stable-read cut, filters replicated to every leg
   ┌───────────────────────────┐
   │ 3  scatter                │  shard 1 … shard N
   └───────────────────────────┘
                 │
                 ▼
   ┌───────────────────────────┐
   │ 4  fuse                   │  reciprocal rank fusion, dedupe by id
   └───────────────────────────┘
                 │
                 ▼
   ┌───────────────────────────┐
   │ 5  cut to pool            │  the L1 stage
   └───────────────────────────┘
                 │ pool candidates
                 ▼
   ┌───────────────────────────┐
   │ 6  rerank      optional   │ ──▶ [ reranker ]
   └───────────────────────────┘     one yes/no relevance question per document
                 │ a calibrated probability per row
                 ▼
   ┌───────────────────────────┐
   │ 7  prune and cut          │  threshold, then top_k
   └───────────────────────────┘
                 │
                 ▼
     rows[] with score  +  routing · plan · hybrid · rerank echo

Stages 1–5 are the gateway’s own work and always run. Stage 6 is the only one that needs a key, and it is the only one you can switch off — with it off, the pipeline ends at stage 5 and score is the fusion rank.

The namespace must declare an embed: attribute and at least one full_text_search attribute.

curl -X POST "$LAYER_GATEWAY_URL/v2/namespaces/papers/search" \
  -H "Authorization: Bearer $LAYER_GATEWAY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "does vitamin D improve bone density in older adults", "top_k": 10}'

Models and keys

Two stages call a model, and they are configured independently.

The query model (stage 1) is a property of the namespace, not of the request: it is whatever the schema’s embed: attribute names, served through that attribute’s serving preference. /search does not choose it and cannot override it. A namespace with several embed: attributes takes embed.attribute to say which one.

The reranker (stage 6) is configured on the gateway, not per request: one OpenAI-compatible endpoint, one key, and a default model. With no key configured the stage is off — see Turning it off.

The default sends the pool to Jev, TypeSafe’s System One model, through OpenRouter — a decision model that answers a yes/no question with a calibrated probability rather than prose, which is exactly the shape stage 6 wants. Nothing about the stage is specific to it. The gateway owns the reranking: it builds the shortlist, writes the relevance question, batches the documents, reads the probabilities back, and prunes. The endpoint is a transport.

So one OpenRouter key reaches every model OpenRouter serves, and the gateway’s endpoint can point at anything else that speaks the same wire — a self-hosted endpoint, another aggregator, a provider’s own API.

Choosing a model per request

rerank.model overrides the gateway’s default model for one request. The key and the base URL are not overridable: a request cannot make the gateway spend a credential on an endpoint the operator did not configure.

{ "query": "...", "rerank": { "model": "typesafe/jev-1.13" } }

The response echoes the model that actually answered under rerank.model, which is not always the slug you asked for — an alias like typesafe/jev-latest resolves to a version.

A model that cannot return a usable probability is a runtime failure, not a validation error: the request degrades and says so.

Turning it off

Three ways, in increasing scope:

Scope	How	Result
One request	`"rerank": false`	Stage skipped. `200`, fused order.
The whole gateway	configure no reranker key	Every request behaves as though it sent `"rerank": false`.
Never quietly	`"rerank": { "required": true }`	A request that cannot rerank fails loudly instead.

With the stage off, the response is still 200 and still complete: rows in fused order, rerank.executed: false with a reason, and score carrying the fusion rank instead of a probability. A gateway with no key answers every request this way and needs no configuration at all.

It is off, not hidden. An unconfigured gateway sets x-layer-warning: rerank_unconfigured on every /search response, so a caller that expected probabilities finds out on the first response rather than from the scores. A caller that will not accept fused-only results sends rerank.required: true and gets 503 rerank_unavailable.

Request

Only query is required. Unknown fields are rejected with 422.

{
  "query": "does vitamin D improve bone density in older adults",
  "top_k": 10,
  "filters": ["year", "Gte", 2015],
  "include_attributes": ["title", "text", "year"],
  "pool": 50,
  "embed": { "attribute": "text" },
  "text": { "fuzziness": "auto", "stopwords": "en" },
  "rerank": {
    "model": "typesafe/jev-latest",
    "threshold": 0.0,
    "attributes": ["title", "text"],
    "docs_per_call": 30,
    "max_chars": 2000,
    "question": "generic-1",
    "required": false
  },
  "explain": false
}

Field	Default	Meaning
`query`	required	The search text. Embedded for the ANN leg, ranked as-is by the BM25 legs, tokenized for the fuzzy legs.
`top_k`	`10`	Rows returned. `1..100`, and at most `pool`.
`filters`	none	Filter expression in the query grammar. A hard filter, replicated to every leg. See Filtering.
`include_attributes`	`true`	`true`, `false`, or a list of attribute names to return under each row’s `attributes`. Reserved `_hevlayer_*` attributes are returned only when named; vector columns are never returned.
`pool`	`50`	Candidates handed to the reranker after the L1 cut. `50..200`. Also sets per-leg depth: `clamp(5 × pool, 50, 200)`.
`embed.attribute`	the only one	Which `embed:` attribute embeds `query`. Required when the schema declares several.
`text.fuzziness`	`"auto"`	`"auto"` or `0..2`, as in `HybridText`.
`text.stopwords`	`"en"`	`"en"`, `false`, or a list. Suppresses fuzzy legs for stop words; BM25 legs are unaffected.
`rerank`	on when a key is set	`false` disables the stage. Otherwise an object; every key is optional.
`rerank.model`	the gateway default	Model slug for this request.
`rerank.threshold`	`0.0`	Drop rows whose probability is below this, before `top_k`. `0.1` is the recommended prune: on BEIR shortlists rows under `0.1` were judged relevant about 0.5% of the time.
`rerank.attributes`	every full-text attribute	Attributes whose text the reranker reads. An attribute named `title` goes first, then schema order. `_hevlayer_*` names are rejected.
`rerank.docs_per_call`	`30`	Documents per reranker call, `1..50`. Calls run concurrently.
`rerank.max_chars`	`2000`	Per-document character budget for reranker text, `1..20000`.
`rerank.question`	`"generic-1"`	Named question version. `generic-1` is the only one.
`rerank.required`	`false`	`true` turns an unconfigured or failing reranker into `503` instead of a degrade.
`explain`	`false`	`true` adds the L1 feature vector to every row.

There is no pagination: a reranked page does not compose with a fused cursor. Ask for a larger top_k, up to pool.

Filtering

filters takes the same expression grammar as /query — no subset, no separate dialect.

What matters is where it applies. The filter is replicated to every leg and pushed into the store, at stage 2, before anything is fused or reranked:

filters: ["tenant", "Eq", "acme"]
        │
        ├──▶ ann leg          WHERE tenant = 'acme'
        ├──▶ bm25:title       WHERE tenant = 'acme'
        ├──▶ bm25:text        WHERE tenant = 'acme'
        └──▶ fuzzy:text:…     WHERE tenant = 'acme'
                              └─▶ fuse ─▶ pool ─▶ rerank

So it is a hard pre-filter, not a post-filter on the returned rows. Two consequences worth relying on:

pool is spent entirely on matching rows. A filter that selects 1% of the namespace still hands the reranker a full pool of candidates from that 1%, rather than a pool drawn from everything and then cut down to a handful.
It is a tenancy boundary, not a display preference. A row excluded by filters is never embedded into a leg result, never fused, never sent to the reranker, and so its text never leaves the environment.

{
  "query": "annual report",
  "filters": ["And", [["tenant", "Eq", "acme"], ["year", "Gte", 2015]]],
  "top_k": 10
}

The filter expression is passed to the store as-is; the gateway does not parse it. A malformed or unsupported expression therefore surfaces as the store’s own error rather than a gateway 422 validation_error.

Response

{
  "rows": [
    {
      "id": "PMC5461234",
      "score": 0.93,
      "attributes": { "title": "...", "text": "...", "year": 2019 }
    }
  ],
  "routing": { "route": "fused", "policy": "v1", "tokens": 10, "executed": true, "advisory": true },
  "plan": { "executed": false, "reason": "unconfigured" },
  "hybrid": {
    "tokens": ["vitamin", "improve", "bone", "density", "older", "adults"],
    "tokens_dropped": 0,
    "stopwords": "en",
    "stopwords_dropped": ["does", "in"],
    "fuzziness": "auto",
    "rank_constant": 60,
    "per_leg_limit": 200,
    "legs": [
      { "label": "ann", "kind": "ann", "attribute": "text", "rows": 200 },
      { "label": "bm25:text", "kind": "bm25", "attribute": "text", "rows": 143 },
      { "label": "bm25:title", "kind": "bm25", "attribute": "title", "rows": 61 },
      { "label": "fuzzy:text:vitamin", "kind": "fuzzy", "attribute": "text", "rows": 88 }
    ],
    "dropped_legs": 0,
    "surfaced": false
  },
  "rerank": {
    "model": "typesafe/jev-1.13",
    "question": "generic-1",
    "executed": true,
    "pool": 50,
    "calls": 2,
    "docs_per_call": 30,
    "threshold": 0.0,
    "pruned": 0,
    "input_tokens": 41200,
    "latency_ms": 231
  },
  "performance": {
    "embedding_tokens": 17,
    "embedding_ms": 4,
    "legs_ms": 88,
    "fuse_ms": 1,
    "l1_ms": 0,
    "rerank_ms": 231,
    "total_ms": 340
  }
}

Every response carries the four echo blocks: routing, plan, hybrid, rerank.

Field	Meaning
`rows[].score`	The reranker’s probability in `[0, 1]` when `rerank.executed` is `true`. It is absolute: comparable across rows, legs, shards and requests, so you can threshold on it. When the stage did not execute, `score` is the fusion sum and is only an ordering. The two are never mixed in one response.
`rows[].attributes`	The attributes `include_attributes` asked for. Text the gateway fetched only for the reranker is stripped.
`routing`	The query router’s decision for this input. `advisory: true`: `/search` runs every leg regardless, and reports the route for the UI and the history.
`plan`	The planner stage. Not configured in this release: always `{"executed": false, "reason": "unconfigured"}`.
`hybrid.legs`	One entry per leg that ran, in fusion order: `label`, `kind` (`ann`, `bm25`, `fuzzy`), the `attribute` it ran over, and the `rows` it returned after the shard merge.
`hybrid.dropped_legs`	Legs the 16-leg budget, or the store, did not run.
`hybrid.surfaced`	`true` when every text leg returned nothing and the fuzzy legs were re-run ordered by edit distance, as in the `HybridText` fallback.
`hybrid.threads`	Shard fan-out width. Present on sharded namespaces only.
`hybrid.fuzziness_clamped`	Present and `true` only when a store forced the effective `fuzziness` to `0`.
`rerank.model`	The model that answered, as the endpoint reports it. Absent when the stage did not execute.
`rerank.executed`	`false` with a `reason` when the stage was off (`disabled`, `unconfigured`), had nothing to score (`empty_pool`), or the call failed (`provider_error`, `timeout`, `rate_limited`).
`rerank.pool`	Candidates scored.
`rerank.calls`	Reranker calls made.
`rerank.pruned`	Rows dropped by `threshold`.
`rerank.input_tokens`	Input tokens across all calls, as the endpoint reports them.
`performance`	Milliseconds per stage, plus the embed wire’s token count.

The response also carries traceparent and, when a stable-read cut applied, x-layer-stable-as-of.

Leg budget

A request runs at most 16 legs. The budget is spent in this order:

The ANN leg.
One BM25 leg per full-text attribute, over the whole query, in schema order.
Fuzzy legs from the HybridText tokenizer policy, one per token, round-robin across attributes, until the budget is spent.

A namespace with many full-text attributes spends its budget on BM25 breadth rather than fuzzy depth. Whatever did not fit is counted in hybrid.dropped_legs.

Explain

With explain: true each row gains an explain object: the feature vector the L1 stage saw and each term’s contribution to the L1 score.

{
  "id": "PMC5461234",
  "score": 0.93,
  "attributes": { "title": "..." },
  "explain": {
    "features": { "rrf_sum": 0.0481, "age_seconds": 86400 },
    "contributions": { "rrf_sum": 0.0481, "age_seconds": 0.0 },
    "l1_score": 0.0481,
    "legs": [
      { "label": "ann", "rank": 3, "score": 0.18 },
      { "label": "bm25:text", "rank": 1, "score": 14.2 }
    ]
  }
}

The L1 stage in this release is the identity: it keeps fusion order and cuts to pool, so every contribution other than rrf_sum is 0. age_seconds is present when the row carries its write stamp; fetch_count_30d is present when the gateway keeps a fetch counter for the namespace. An absent feature is absent, not 0.

Degrade

If the reranker errors, times out, or keeps answering 429, the response is still 200: rows in fused order, score as the fusion sum, rerank.executed: false with a reason, and the header x-layer-warning: rerank_degraded. Set rerank.required: true to get 503 rerank_unavailable instead.

Degrade is the same shape as running with no key at all, which is the point: a client written against the fused response works in both, and the warning header is the only thing it has to read to tell them apart.

Errors

Status	`error`	When
`422`	`embed_attribute_missing`	The schema declares no `embed:` attribute.
`422`	`embed_attribute_invalid`	`embed.attribute` names an attribute without `embed:`, or the schema declares several and none is named.
`422`	`full_text_attribute_missing`	The schema has no `full_text_search` attribute.
`422`	`UnsupportedByStore`	The store cannot serve an ANN or a BM25 leg. See Stores.
`422`	`validation_error`	`top_k > pool`, `pool` out of range, unknown question, unknown field, a query that yields no tokens.
`503`	`rerank_unavailable`	`rerank.required` is `true` and the stage is unconfigured or the call failed.

Stores

/search executes on turbopuffer, where reranker text is returned by the store and billed as returned bytes. A store that cannot serve an ANN or a BM25 leg answers 422 UnsupportedByStore.

Data leaves the environment

Candidate text is sent to the reranker endpoint, and query is sent to the embedding provider when the embed: attribute serves natively on turbopuffer rather than locally. Configuring no reranker key, or sending rerank: false, removes the first entirely. Reserved _hevlayer_* attributes are never a text leg and never reranker input. A reranker endpoint inside your own network keeps candidate text there.

History

Each request writes one search history entry with raw_query taken from the body, so no x-hevlayer-search-query header is needed.