Performance & Latency

p95 Latency

Measured policy-layer overhead: 16 ms p50, 21 ms p95 above the raw vector DB call (benchmark of 2026-08-16). Methodology: a pgvector baseline with 1,000 vectors, a 5-rule RBAC policy, 100 registered resources, sidecar metadata resolution, and client-supplied query embeddings, over repeated sequential runs. The underlying data is public: the headline run is at policy-overhead-bench-2026-08-16.json, and all five recorded runs are at policy-overhead-bench-all-runs.json. Every retrieval records the same split (total latency vs connector latency) in its audit record. The overhead covers principal load, metadata resolution, policy evaluation, audit and usage writes, and response assembly. HTTP middleware and response serialization add roughly 30 ms at p50 and 45 ms at p95 more end to end (full series in the artifact); the raw vector query itself is excluded from both figures.

You can measure this on your own account rather than taking ours. Every plan, including the free one, has a Performance Self-Test page that reports the policy-layer overhead recorded on your own retrievals, which is the same quantity as the figure above. It also runs the policy engine live against your own active policies. Running it does not consume your retrieval quota, because it calls neither a vector database nor an embedding provider. Sign in and open Performance Self-Test under Observe.

What We Cache (and Don't)

Gateco deliberately caches almost nothing on the retrieval path. Policies, principal groups and attributes, and resource metadata are read live on every request, so a policy change or a directory change takes effect on the next retrieval rather than when a cache expires. The measured overhead figures above are achieved with these live reads included.

Throughput

A single backend pod is sized for roughly 200 RPS (engineering estimate; not yet benchmarked). Scale horizontally. All state is in Postgres and workers are stateless. The auto-sync scheduler uses PostgreSQL advisory locks to coordinate across instances without double-execution.

Connector Latency

Vendor-typical p95 for a warm vector query in the same region, taken from vendor documentation and benchmarks. We have not benchmarked these per connector, and they exclude connection setup. For calibration: our own published benchmark measured the full local pgvector leg at ~126 ms p50, dominated by per-request connection setup (a per-DSN connection pool is the filed fix). Treat the table as vendor guidance, not a Gateco measurement.

Getting Better Performance

• 1.

  Use sidecar metadata resolution (default)

  Sidecar mode reads metadata from Gateco's own registry, one indexed Postgres lookup per retrieval. Inline and sql_view modes add an extra network round-trip to the connector.

• 2.

  Keep policy rule count under 20 per policy

  Policy evaluation cost is linear in rules and candidates. In the published benchmark, evaluating a 5-rule policy over ~30 candidates took about a millisecond. Above 50 rules, consider splitting into multiple narrower policies.

• 3.

  Supply your own query embeddings

  Requests that include a query_vector skip the server-side embedding call, so the embedding provider's latency stays out of your retrieval path entirely. This is also how the published benchmark isolates policy overhead.