Configuration reference#
Every option each connector takes, with its default, in one place per connector.
| Page | Surface |
|---|---|
| BigQuery | The source and sink builders, bounded table and query reads, the three write methods’ options objects, experimental CDC writes and table provisioning, table creation and schema updates, and serializer schema options |
| Cloud Pub/Sub | The sink and source builders, publisher and subscriber tuning, topic and subscription creation |
| Cloud Tasks | The sink builder, writer concurrency and retry budgets, and staged recovery settings |
| Bigtable | The sink builder and the writer’s batch thresholds and in-flight bounds, and the source builder’s ranges, prefixes and filter |
| Spanner | The sink builder, the writer’s batch limits, commit delay, RPC priority and retry budget |
| Firestore | The source builder’s read shape, partitions, read time and page size; the sink builder, the client library’s throttle and retry budget, and the writer’s in-flight bounds; the Datastore-mode source builder’s read shape, namespace, split count, read time and page size; the Datastore-mode sink builder, its batch bounds, retry budget and ramp-up throttle |
These pages answer what; the connector pages answer why. A row here gives you the option’s
name, its default and one line on what it does. The reasoning behind a default — why
maxInflightRequests departs from the SDK’s own, why ordering is off, why the Cloud Tasks sink has
no rate knobs at all — stays on the /flink-connector-gcp/1.2/docs/connectors/ page it belongs to, and each
section below links to it.
Two surfaces are documented elsewhere, deliberately. The SQL connector pages —
BigQuery,
Pub/Sub,
Cloud Tasks,
Bigtable and
Spanner — carry the full WITH option
surface: most SQL options exist where a builder setter does and the mapping between them is
the thing worth reading, and the table-owned selectors with no builder counterpart — a
scan.mode, a schema mapping — are documented nowhere else. And every type named here — including the enums, the destination types and the
SPIs an option takes — is in the Java API reference, generated from the
source.
Duration validation#
Duration settings with a one-millisecond minimum reject positive values below it.
Settings whose SDK gives zero a separate meaning may also accept Duration.ZERO.
An option may have a tighter limit, such as the nanosecond budget bound documented on its setter, and that check may reject the value first with its own diagnostic.
If validation reaches the one-millisecond check, a millisecond conversion overflow is rejected with an IllegalArgumentException naming the option and the input value.
The conversion’s ArithmeticException remains its cause.
Through a Table API option mapper, a setter’s IllegalArgumentException becomes a ValidationException naming the SQL option key.
What a default means#
Three kinds of entry appear in the Default column, and the difference matters:
| Entry | Meaning |
|---|---|
| A value | The connector’s own default, applied when you do not set the option |
| required | The builder rejects a sink or source built without it |
| unset ⇒ … | The connector sets nothing, so the client library’s or the service’s own default applies. The named value is that default, recorded here for sizing rather than enforced by this project — it can change under a dependency bump |
The last row is why an options object’s defaults() is equivalent to not passing one at all: it
holds no values of its own for those knobs.