Skip to main content

APIs

Output from the processor is mapped straight into a product’s APIs: the usual pattern is key–value-style access backed by materialized state, with RPC-style operations as a more niche layer on top. Both are declared in Spine and ship through the Data Product Manifest.

  • Key–value APIs (typical case) — Address data by key (and related patterns your product defines) with very fast reads because results are materialized for the API as the pipeline runs. The store can be walked or iterated on the data-product side for operations that need it.
  • RPC APIs (specialized case) — Request/response calls with arbitrary logic. They can read the same key–value materializations internally and add further processing, or call back into the platform (for example the control plane). The Admin API is itself implemented as a data product using that pattern: unary RPCs that trigger internal platform work.

Ingests and outlets still cover high-volume append and change streams; APIs are the place for discrete calls against the product’s current contract (dashboards, tools, and synchronous integration).

Consistency: ingest, processing, and what you read​

Ingesting a message and immediately seeing the outcome on an API is not a single atomic step. The processor consumes the ingest, and backlog or load affects how long it takes before a value shows up in the materialized view you query. The pattern is CQRS-like: commands (ingest) and queries (the API) are separated; the read side is a materialized projection updated by the processor, so read-your-write on the API only holds once your event has been processed into that view.

Queries, filtering, and subscriptions​

Rich or ad-hoc filtering and analytical querying are out of scope for the platform: model what you need in the product, then index or search in your own systems on the consumer side. API subscriptions (subscribe to a current value and subsequent changes) are on the roadmap; today, change propagation for keys is the outlet path.

Large and binary payloads​

The platform is built for large payloads, including binary assets (for example PDFs, images, or audio). When the data product exposes an operation for such content (often marked supports_raw_response in the manifest), REST offers a path that returns the bytes without the usual API envelope when you negotiate the right Accept—useful to stream to another system, write a file, or hand the body to a browser. See Raw response operations. gRPC unary calls use the ApiResponse wrapper; choose REST when you need a bare HTTP body for that media type.

How to call APIs​

Client SDKs and the application integration path are the recommended way to get typed, maintained behavior. The same operations are also available as HTTP (see REST) and as MCP tools for AI agents and assistants, alongside gRPC and encodings in the transports reference.

See also — Ingests · Outlets · Processor · Client SDKs · MCP · Control plane