Table API Connectors#
Use these pages for DDL, SQL type mappings, metadata columns, and planner-specific restrictions.
Each Table connector maps onto the DataStream connector it is documented with rather than providing a
separate implementation; the datastore connector maps onto the Firestore connector’s Datastore mode.
The corresponding DataStream connector page
documents runtime behavior and builder-only features.
The connector overview explains how to choose between the two
APIs.
Capability map#
Pushdown describes bounded scans; the lookup column separately describes point-read behavior.
Metadata stays with the direction that reads or writes it.
A catalog lists a service’s tables and derives each schema, in place of a hand-written CREATE TABLE.
Not applicable means that the capability does not fit the service’s role; it is not an
implementation-status claim.
| Connector pages | Primary role | Scan and pushdown | Sink changelog | Lookup | CDC composition | Metadata | Catalog |
|---|---|---|---|---|---|---|---|
| BigQuery: Quickstart, Table examples source, sink, and CDC, Table reference source, sink, and CDC, DataStream reference source and sink | Analytics warehouse | Bounded; projection and filter pushdown | Insert or keyed CDC upsert/delete; Experimental (#706) | Not supported by the bounded Storage Read source | Consumes an upsert/delete changelog | Sink-writable CDC sequence inputs; no readable metadata | Read-only; datasets as databases and tables resolved from their BigQuery schemas (catalog) |
| Cloud Pub/Sub: Quickstart, Examples, DataStream (source, sink), Table (source, sink) | Messaging | Unbounded | Insert | Not applicable | Carries a format-provided source changelog | Readable and writable message metadata | Not applicable; the connector reads no topic schema, and a topic without one has no columns to derive |
| Cloud Tasks: Quickstart, Examples, DataStream, Table | Task delivery | Not applicable; sink only | Insert; opt-in checkpointed creation within its recovery window | Not applicable | Not applicable | Writable task and request metadata | Not applicable; sink only, and a queue carries no schema |
| Bigtable: Quickstart, Examples, DataStream source, sink, and Change Streams, Table source, sink, lookup, and Change Streams | Wide-column store | Bounded; projection and row-key filter pushdown | Upsert and atomic keep-latest; insert-only compatibility mode; atomic insert-if-absent and DDL-defined conditional commands with INSERT-only input | Sync or async with partial cache; synchronous full cache; projected families | Envelope for transformation, or keyed upsert/delete directly to an upsert sink | Readable CDC and writable cell-timestamp metadata | Read-only; an instance as one database and its tables resolved as a row key and one map per column family (catalog) |
| Spanner: Quickstart, Examples, DataStream, Table | Relational database | Bounded; projection and primary-key filter pushdown | Insert or upsert | Sync or async; primary-key filter pushdown; partial cache | Full retract or keyed upsert/delete source to keyed sink | Readable CDC transaction metadata | Read-only; an instance’s databases as databases and base tables resolved from INFORMATION_SCHEMA in both dialects (catalog) |
| Firestore: Quickstart, Examples, DataStream, Table | Document database | Bounded snapshot of one collection or a collection group; projection pushdown | Insert as new documents, or keyed upsert/delete (merge by default, replace) or upsert (update); same-document order not kept | Document id only; sync or async; NONE or PARTIAL cache | Consumes a keyed upsert/delete changelog, without deletes under update | Readable document path and create, update and read times | Not applicable; a collection carries no schema |
| Firestore in Datastore mode: DataStream, Table | Document database | Bounded snapshot of one kind in key ranges; column projection client-side | Insert as new entities, or keyed upsert/delete (merge by default, replace) or upsert (update); one key’s order kept per subtask | Key name or id only; sync or async; NONE or PARTIAL cache | Consumes a keyed upsert/delete changelog, without deletes under update | Readable key name or id, version, and create, update and read times | Not applicable; a kind carries no schema |
The Bigtable and Spanner keyed source-to-sink paths describe compatible changelog shapes, not a
stronger replication guarantee.
For example, a Spanner Change Streams table in upsert mode can feed a separate primary-key
Spanner sink table, but that sink remains at-least-once and does not guarantee the application
order of successive writes to one key.