Query gateway flow

Query gateway flow A data-flow diagram generated by Archify. 01 / Clients + identity 02 / Admit 03 / Plan 04 / Prune + scan 05 / Your storage + catalog OIDC provider · signing keys (JWKS) · 01 / Clients + identity OIDC provider signing keys (JWKS) Clients · HTTP · PostgreSQL · 01 / Clients + identity Clients HTTP · PostgreSQL Authenticate · token · dataset role · 02 / Admit Authenticate token · dataset role Validate + plan · names · window cap · 03 / Plan Validate + plan names · window cap Snapshot cache · poll every 8 seconds · 04 / Prune + scan Snapshot cache poll every 8 seconds Prune files · stats · bloom · text · 04 / Prune + scan Prune files stats · bloom · text Scan + stream · only surviving files · 04 / Prune + scan Scan + stream only surviving files Iceberg catalog · current snapshot · 05 / Your storage + catalog Iceberg catalog current snapshot Azure Blob · data files + sidecars · 05 / Your storage + catalog Azure Blob data files + sidecars query fetch keys allowed filter poll file list sidecars survivors range reads rows, streamed Legend query path authentication background poll response

Admit

  • • Every request needs a valid OIDC bearer token
  • • With dataset roles set, other datasets are denied

Prune

  • • Column statistics, then bloom, then text index
  • • A file with no sidecar is scanned, not skipped

Limits

  • • A raw query spans 7 days at most, by default
  • • Rows stream; memory does not grow with the result