All Classes and Interfaces

Class
Description
Keeps a flink-sql-connector-gcp-* module's META-INF/NOTICE honest: it must list every artifact the uber-jar bundles, and nothing it does not.
Asserts the shape of a flink-sql-connector-gcp-* uber-jar: that SQL can discover the connector in it, that every runtime dependency actually made it in, and that no class escaped relocation except those that deliberately did.
The precondition every flink-sql-connector-gcp-* smoke test rests on: that the connector it just exercised came out of the uber-jar.
Settles one delivered message, hiding which of the client library's two receiver flavors produced it.
Tracks the acknowledgement lifecycle of received Pub/Sub messages, which is what makes the source at-least-once.
One physical BigQuery column appended after the configured serializer emits a row.
Determines how an additional physical field handles a provider returning null.
Ordered physical fields appended to every destination row and table schema.
Builder for AdditionalFields.
Supported BigQuery types for singular additional physical fields.
Extracts one additional physical-field value from the original input record.
Classifies Storage Write API append failures into the error classes the writer routes on.
Extracts App Engine request headers from a record.
Extracts the App Engine relative URI from a record.
Extracts App Engine service, version, and instance routing from a sink record.
Checks and normalizes App Engine request fields shared by all sink APIs.
A CloudTasksSerializationSchema producing App Engine target tasks.
Validated fixed App Engine request values used by the table serializer.
Converts Avro records into the protobuf rows the BigQuery Storage Write API accepts, against the schema AvroToTableSchemaConverter derived and the descriptor BQTableSchemaToProtoDescriptor built from it.
Options controlling how Avro schemas are mapped to BigQuery schemas.
Builder for AvroSchemaOptions.
Derives a BigQuery TableSchema from an Avro Schema.
Deadline-bounded polling for test assertions.
Plans a read through a google-cloud-spanner BatchClient.
Reads partitions through a google-cloud-spanner BatchClient.
One partition of a batch read, together with the snapshot it belongs to.
Serializer for BatchReadSplit.
The mutable reader-side state of a BatchReadSplit.
The WriteMethod.STORAGE_API_EXACTLY_ONCE sink: each writer subtask appends rows to one application-created BUFFERED Storage Write API stream per active destination at explicit offsets (see BigQueryBufferedStreamWriter), and the committer makes each completed checkpoint's rows visible by flushing every committable's stream up to its offset (see BufferedStreamCommitter) — two-phase commit on Flink checkpoints.
Exactly-once SinkWriter appending rows to one application-created BUFFERED Storage Write API stream per destination at explicit offsets.
The WITH options of the bigquery table connector.
Loads credentials shared by the BigQuery and Cloud Storage client families.
At-least-once sink appending to Storage Write API default streams with dynamic per-record table destinations (WriteMethod.STORAGE_API_AT_LEAST_ONCE).
At-least-once SinkWriter appending to Storage Write API default streams with dynamic per-record table destinations.
The bigquery table sink.
Collects the sink's fully resolved values.
A bounded Table API source mapped onto BigQuerySource.
Collects the source's fully resolved values.
Creates the bigquery table source or sink from a CREATE TABLE statement's options.
The BigQuery emulator image and its two endpoints, shared by every harness that starts the emulator so they cannot drift apart.
A record-specific BigQuery sink failure handled by the configured failure policy.
The WriteMethod.FILE_LOADS sink: writers stage per-destination files on Cloud Storage in the configured staging format, the pre-commit topology routes every subtask's committables to a single committer subtask (stamping their checkpoint id in streaming, see FileLoadsCheckpointStamper), and the committer turns each batch into BigQuery load jobs (see FileLoadsCommitter and LoadJobOrchestrator) — the whole run at end of input in batch execution, each checkpoint's files at its completion in streaming execution.
LoadJobRunner over the BigQuery REST client with configured credentials or ADC.
Thread-safe submitted-job state shared by every worker of one committer.
Every metric name this connector registers itself, in one place so that this file is the connector's inventory: what it reports can be read here without opening a writer.
Serializes records into protobuf rows to be written to BigQuery.
Runs the source's query through the REST BigQuery client.
Storage Read API clients, for the two places the source opens one: the enumerator creates the read session, and every reader opens its assigned streams.
The enumerator's checkpointed state: whether the read session exists, which one it is, and the splits not currently assigned to a reader.
Creates the read session once and hands its streams out one at a time.
Converts each row and advances the stream's offset by exactly one row after successful deserialization and downstream collection.
Converts a row read from the BigQuery Storage Read API into zero or more output records.
Converts a REST client Schema (as read from live tables) into the Storage API TableSchema form the sink's schema handling operates on.
Entry point for building a BigQuery sink.
Builder for BigQuery sinks, obtained from BigQuerySink.builder().
Immutable configuration shared by all WriteMethod implementations, assembled by BigQuerySinkBuilder.
A bounded source reading a BigQuery table through the Storage Read API.
Builds a BigQuerySource.
Everything the source's enumerator and readers were configured with, as one immutable object shipped inside the job graph.
Reader subtask consuming the read streams it is assigned, one at a time.
The reader's metrics.
Reads the rows of the streams this subtask was assigned, a bounded batch at a time.
Reads a BigQuery table through the Storage Read API.
Default TableAdmin backed by the BigQuery REST client.
Turns one Bigtable change-stream mutation into zero or more user records.
Table changelog source backed by the DataStream Change Streams source.
Checkpointed coordinator state for the Bigtable Change Streams partition topology.
Connector-owned checkpoint format for BigtableChangeStreamEnumeratorState.
The exact physical schema of the generic Bigtable Change Streams mutation envelope.
One complete logical row mutation returned by Bigtable Change Streams.
Adds an aggregate input to one cell.
How a BigtableChangeStreamMutation.TimestampBound constrains its end of the range: exclusive, inclusive, or not at all.
Deletes cell versions in one timestamp range.
Deletes every cell in one family.
One ordered mutation entry.
A signed 64-bit integer in an aggregate entry.
Merges an aggregate input into one cell.
What produced a mutation: a user write, or the table's garbage-collection policy.
A raw microsecond timestamp in an aggregate entry.
Arbitrary bytes in an aggregate entry.
Writes one cell version.
One timestamp-range boundary.
Timestamp range affected by a cell deletion.
One typed value used by an aggregate entry.
Emits the connector-owned BigtableChangeStreamMutation unchanged.
The connector-side family and qualifier patterns, answering whether one identifier is retained.
Serializes the connector-owned mutation model without reflective collection access.
Snapshot for the connector-owned field format.
Supplies the connector-owned serializer for BigtableChangeStreamMutation.
Reads a bounded number of Bigtable Change Streams partitions concurrently.
Reporter-visible state for one Bigtable Change Streams reader subtask.
Emits mutations with commit timestamps and advances token/watermark state.
FLIP-27 source for Bigtable Change Streams.
Everything the Change Streams source was built with, assembled by the builder and carried into the job graph.
Coordinates Bigtable Change Streams partitions, split successors, and merge-parent tokens.
The WITH options of the bigtable table connector.
Loads credentials for the Bigtable data and admin client families.
How this connector points a BigtableDataClient at an instance: production over application-default credentials, production over a runtime-loaded service-account provider, or an emulator over a plaintext channel with no credentials.
The bigtable table sink.
Collects the sink's values, so no caller has to keep a positional list in order.
The bigtable table source: a bounded scan over the DataStream BigtableSource and row-key point lookups through Bigtable's data client.
Collects the source's values, so no caller has to keep a positional list in order.
Creates the bigtable table sink and table source from a CREATE TABLE statement's options.
The Bigtable emulator image shared by every harness that starts the emulator, so they cannot drift apart.
The lookup values a CREATE TABLE maps into the Bigtable table source.
Every metric name this connector registers itself, in one place so that this file is the connector's inventory: what it reports can be read here without opening the writer.
At-least-once sink applying one row mutation per record through the MutateRows bulk batcher of google-cloud-bigtable.
Deserializes each row and records that the split has passed it.
An asynchronous Bigtable row-key point lookup producing the table source's projected row.
A synchronous Bigtable row-key point lookup producing the table source's projected row.
Turns a Bigtable row into the records a job works with.
What the split enumerator checkpoints: whether the table has been sampled and planned, and the splits nobody is holding yet.
The bounded scan source over Bigtable's ReadRows.
Samples the table once, cuts the configured ranges at the sampled boundaries, and hands the pieces out one at a time.
Serializes sink records into Bigtable row mutations.
Entry point for building a Bigtable sink.
Builder for Bigtable sinks, obtained from BigtableSink.builder().
Immutable sink configuration assembled by BigtableSinkBuilder.
Reads rows from a Cloud Bigtable table.
Builds a BigtableSource.
Everything the scan source was built with, assembled by the builder and carried into the job graph.
Reads the splits this subtask is assigned, asking the enumerator for the next one each time it finishes one.
The scan reader's own counters, beside the ones Flink registers for every source.
Reads the row-key ranges this subtask was assigned, a bounded batch at a time.
Default TableAdmin backed by the Bigtable BigtableTableAdminClient.
The DDL schema of a bigtable table: which column is the row key, and which column family and qualifier every other column addresses.
One column family: a ROW column whose nested fields are its qualifiers.
One qualifier: a nested field of a column family's ROW.
Stock Bigtable clients pointed at an emulator container.
At-least-once writer applying row mutations to the Bigtable tables a DestinationResolver names.
Tuning options for the sink's writer: the batch thresholds handed to the client, and the writer's own bounds on unacknowledged entries.
The teardown of one client whose own shutdown cannot be trusted to return: both of its steps on a separate thread, and one deadline that the calling thread's single join is the whole of.
The client's own bounded wait, satisfied by e.g.
A flush instruction for one buffered write stream: make every row up to flushOffset (inclusive, per FlushRows semantics) visible in the destination table.
Committer of the buffered-stream write path: makes each completed checkpoint's rows visible by flushing every committable's stream up to its offset (FlushRows).
Options specific to WriteMethod.STORAGE_API_EXACTLY_ONCE: how large append requests may grow, how long inactive destinations stay cached, how the connector-driven recovery schedule backs off, and how the SDK retries retriable append failures in-stream.
Maps the sink.buffered-stream.* options onto BufferedStreamOptions.
The Storage Write API operations of the buffered-stream write path, abstracted so writer and committer logic can be unit-tested against fakes: creating buffered streams, opening offset-aware appenders on them, and flushing rows.
Creates BufferedStreamService instances on the task manager.
One destination's buffered-stream writer state: which stream the subtask owns and how many rows it had appended (acknowledged) as of the checkpoint.
What one subscriber holds buffered and has not yet handed to the reader, as a single consistent reading.
A mutation applied by BigQuery change data capture.
Determines whether one record upserts or deletes its BigQuery primary key.
Options for appending BigQuery CDC mutations through the Storage Write API default stream.
Builder for CdcOptions.
Creates the internal write-only fields that represent BigQuery CDC pseudocolumns.
Extracts the BigQuery change sequence number for one record.
Desired BigQuery table contract for CDC writes.
Builder for CdcTableOptions.
Maps Table API CDC table options onto their DataStream contract.
Supplies the desired CDC table contract for each destination.
Policy for a BigQuery CDC destination table that already exists.
Turns a RowData field into the bytes of a Bigtable cell, and back.
Reads the bytes of a cell as one field of a row.
Reads one field of a row as the bytes of a cell.
How many cells a mutation costs, counted the way Spanner counts a mutation.
The physical changelog representation emitted by a Bigtable Change Streams table source.
The relational changelog emitted from Spanner data-change records.
Coordinator-side Bigtable operations, separated so partition protocol tests need no service.
Mints the ChangeStreamCoordinatorClient one enumerator coordinates through.
Dispatches a change-stream mutation entry or aggregate value to the connector code that handles it.
Handles every mutation entry type the connector models.
Handles every aggregate value type the connector models.
Serializable seam that opens asynchronous ReadChangeStream RPCs.
The shape every Bigtable Change Streams partition has, and the conversion to the shape the SDK models require.
One Bigtable Change Streams partition and the exact position from which it resumes.
One Spanner Change Streams partition and its checkpointed coordinator lifecycle.
Connector-owned checkpoint format for ChangeStreamPartitionSplit.
Connector-owned checkpoint format for a Spanner Change Streams partition split.
Mutable task-thread state for one change-stream partition.
Mutable task-thread progress for one Spanner Change Streams partition.
Resolves a reader-restored split against current retention.
A table option's Bigtable Change Streams start-position mode.
A Table API spelling for one shared Change Streams start position.
Maps one mode/timestamp option pair onto the shared Change Streams start-position API.
Maps Table API startup options to the shared change-stream start position.
One child-partitions record discovered while a reader is consuming a parent partition.
One child token and the normalized coordinator identities of every parent.
Releasing several resources at once, so that one refusing to close never strands the rest.
The WITH options of the cloud-tasks table connector.
At-least-once sink creating one Cloud Tasks task per record through the CreateTask RPC, with fixed or per-record queue destinations.
Insert-only table sink creating target tasks in one fixed queue.
Factory for the sink-only cloud-tasks table connector.
The Cloud Tasks emulator image shared by every harness that starts the emulator, so they cannot drift apart.
Every metric name this connector registers itself, in one place so that this file is the connector's inventory: what it reports can be read here without opening the writer.
Serializes sink records into Cloud Tasks tasks.
Entry point for building a Cloud Tasks sink.
Builder for Cloud Tasks sinks, obtained from CloudTasksSink.builder().
Immutable sink configuration assembled by CloudTasksSinkBuilder.
The request target represented by a cloud-tasks table.
A stock CloudTasksClient over one plaintext channel to the emulator, with no credentials, for queue administration and task inspection.
At-least-once writer creating one Cloud Tasks task per record.
Tuning options for the sink's writer: the in-flight cap, the transport channel pool and the two retry budgets.
ReaderOutput for tests that drive SourceReaderBase.pollNext(...), delegating to one shared CollectingSourceOutput whatever split a record came from.
SourceOutput recording emitted records and their timestamps — what a RecordEmitter is handed, and what a test driving one asserts against.
What the sink does with a mutation Spanner refuses for violating a constraint.
Everything one BigQuery copy job needs, decoupled from the client for testability.
Whether the sink may create destination tables that do not exist.
Whether the sink may create the destination table when it does not exist.
Whether the sink may create destination topics that do not exist.
The cross-version seam every sink in this module implements instead of Sink directly.
The cross-version seam every sink in this module implements instead of Sink directly.
The cross-version seam every sink in this module implements instead of Sink directly.
The cross-version seam every sink in this module implements instead of Sink directly.
The cross-version seam every sink in this module implements instead of Sink directly.
A fully-qualified Spanner database reference: project, instance and database.
One data-change record returned by a Spanner Change Streams read function.
Builder for DataChangeRecord.
Per-record description of one watched column.
Serializes DataChangeRecord without reflective access to JDK implementation fields.
Snapshot for the connector-owned field format.
Supplies the connector-owned serializer for DataChangeRecord.
Opens change streams through a lazily created Bigtable data client.
Samples row keys through a google-cloud-bigtable BigtableDataClient.
Opens reads through a google-cloud-bigtable BigtableDataClient.
Destination for elements that terminally failed to be written by a sink, used through FailureHandler.sendToDeadLetterQueue(DeadLetterQueue).
Transcodes Debezium MySQL GTID coordinates into BigQuery CDC sequence sections.
Derives BigQuery CDC sequence numbers from Debezium MySQL source properties.
Transcodes Debezium PostgreSQL sequence metadata into BigQuery CDC hexadecimal sections.
Derives BigQuery CDC sequence numbers from Debezium PostgreSQL source properties.
Derives BigQuery CDC sequence numbers from Debezium Spanner source properties.
Native java-bigtable implementation of the coordinator operations.
Retention-aware reader restore resolution backed by Bigtable table metadata.
The FailureHandlerContext the sinks build from their WriterInitContext.
Creates MutationBatchers backed by google-cloud-bigtable BigtableDataClients, connected the way BigtableDataClients connects every data client of this connector — that class owns the credential and emulator modes.
PublisherFactory building google-cloud-pubsub Publisher instances configured from PubSubPublisherOptions; every knob left unset keeps the SDK default (options with no batching or retry overrides leave the publisher builder untouched, so the default configuration is byte-identical to the SDK's).
Opens the Spanner service handle used for change-stream metadata discovery.
Opens the Spanner client and bounded callback executor owned by one reader subtask.
Opens a real Spanner service handle and wraps it in a SpannerServiceAdapter.
Options specific to WriteMethod.STORAGE_API_AT_LEAST_ONCE: how large append requests may grow, how the connector-driven re-append budget backs off, how the SDK retries retriable append failures in-stream, how the SDK's connection pool (multiplexing) is sized, and the writer's housekeeping (destinationIdleTimeout, flushInterval).
Maps the sink.default-stream.* options onto DefaultStreamOptions.
SubscriberFactory building google-cloud-pubsub Subscriber instances configured from PubSubSubscriberOptions; every knob left unset keeps the SDK default, so the default configuration differs from the SDK's only in the two settings the source owns.
Creates a TaskCreator backed by a google-cloud-tasks CloudTasksClient, connecting either to production Cloud Tasks with application-default or configured service-account credentials, or to an emulator over a plaintext channel with no credentials.
What the source does with a message its deserialization schema cannot convert.
Runs independent destination actions with a fixed, committer-lifetime concurrency bound.
One worker's thread-confined BigQuery collaborators.
Builds one worker context, lazily on the thread that owns it.
Optional per-destination send counters, as destination.NAME.recordsSend and destination.NAME.sendErrors on the writer's metric group, where NAME is the destination as the connector's docs page spells it.
One destination's counters; a no-op instance when per-destination metrics are off.
The result of resolving a BigQuery destination for one record.
Dispatches destination-resolution results to the writer that owns the record.
Handles every destination-resolution result supported by the connector.
Resolves the destination table for each record, enabling one sink instance to write to many tables (dynamic destinations).
Resolves the destination table for each record, enabling one sink instance to write to many tables (dynamic destinations).
Resolves the destination queue for each record, enabling one sink instance to write to many queues (dynamic destinations).
Resolves the destination topic for each record, enabling one sink instance to publish to many topics (dynamic destinations).
Deadline-bounded draining of a running job's collect iterator.
Transport channel providers pointing a gRPC client at an emulator.
The host:port of a Google Cloud emulator, parsed and validated on the client rather than on a TaskManager after the job has been submitted (issue #235).
The three counters a PullAssignmentSplitEnumerator reports, registered by the connector under its own names.
Counts a sink's failures by the status code that caused them, as errorClass.CODE.errors on the writer's metric group.
An element that terminally failed to be written by one of this project's sinks — the read-only contract every connector's concrete failure type implements, and the view a cross-connector DeadLetterQueue sees.
A single message that terminally failed to be published to Pub/Sub, as passed to a FailureHandler<FailedMessage>.
A single row mutation that terminally failed to be written to Bigtable, as passed to a FailureHandler<FailedMutation>.
A single mutation that terminally failed to be applied to Spanner, as passed to a FailureHandler<FailedMutation>.
A single row that terminally failed to be written to BigQuery, as passed to a FailureHandler<BigQueryFailure>.
A single task that terminally failed to be created in Cloud Tasks, as passed to a FailureHandler<FailedTask>.
Pluggable policy for elements that terminally fail to be written by a sink.
What a FailureHandler or DeadLetterQueue learns about its surroundings when it is opened: enough to stamp per-subtask output and to register its own metrics, and nothing more.
In-memory SourceReaderContext for the tests of a pull-assignment source reader.
FakeSplitEnumeratorContext<SplitT extends org.apache.flink.api.connector.source.SourceSplit>
In-memory SplitEnumeratorContext for the tests of a pull-assignment enumerator — one whose readers ask for work through handleSplitRequest.
The streaming pre-commit stage of the FILE_LOADS topology: stamps each committable with the checkpoint id carried by its CommittableMessage, which the Committer SPI cannot observe otherwise — the stamp selects the streaming behavior of the load-job orchestrator and makes BigQuery job ids attributable to their checkpoint.
One finalized staging file: the Flink job id of the run that staged it, its destination table, its Cloud Storage URI, size counters used for load-job partitioning, and — in streaming execution — the checkpoint that triggered it.
Serializer for FileLoadsCommittable.
The real FILE_LOADS commit: turns one committable batch — a whole batch run, or one checkpoint of a streaming run — into BigQuery load jobs through a LoadJobOrchestrator.
Thread-safe metrics for the bounded FILE_LOADS destination executor.
Options specific to WriteMethod.FILE_LOADS: where staging files go on Cloud Storage, how loaded rows land in tables that already hold data, where oversized loads stage their temporary tables, how checkpoint-triggered loads are paced in streaming execution, and the two schedules the committer backs off on.
Builder for FileLoadsOptions.
Maps the sink.file-loads.* options onto FileLoadsOptions.
The FILE_LOADS SinkWriter: encodes records in the configured staging format and streams them to per-destination staging files on Cloud Storage, emitting one FileLoadsCommittable per finalized file from FileLoadsWriter.prepareCommit() — once at end of input in batch execution, once per checkpoint in streaming execution.
A DestinationResolver returning one fixed destination for every record.
A DestinationResolver returning one fixed destination for every record, which is what BigtableSinkBuilder.table(...) builds.
A DestinationResolver returning one fixed destination for every record.
A DestinationResolver returning one fixed destination for every record.
Table format factory for UTF-8 application/x-www-form-urlencoded bodies.
A garbage-collection rule for a column family the sink creates, mirroring the four rule shapes Bigtable's admin API takes: a version cap, an age cap, and their union and intersection.
The rule shapes Bigtable's admin API takes.
StagingStorage over the Cloud Storage client with configured credentials or ADC.
An encoding format that owns the HTTP Content-Type for body-carrying requests.
The first stage of CloudTasksSerializationSchema.httpTarget(String): it carries the target URL and waits for the body schema.
A CloudTasksSerializationSchema producing HTTP-target tasks: a URL, an HTTP method, an optional authorization token, optional headers, and a body serialized by a wrapped Flink SerializationSchema.
Extracts HTTP headers from a record.
Extracts the target URL from a record.
Validated fixed HTTP request values used by the table serializer.
What the Bigtable table sink tells the planner when the requested changelog is insert-only.
Options controlling how JSON records are converted to BigQuery rows.
Builder for JsonDocumentOptions.
A BigQueryProtoSerializationSchema for records that are JSON documents, as text.
A BigtableDataClient built on first use and closed once, held by a seam implementation that travels in the job graph.
Holds state its owner derives from its own serializable fields, once, and again on a task manager.
Runs the staged files of one run — a whole batch job, or one checkpoint of a streaming job — into their destination tables: submits every load, copy and terminal query through a LoadJobRunner, then deletes the temporary tables and the staged objects.
Executes BigQuery load, copy and terminal query jobs for the FILE_LOADS orchestration.
Everything one BigQuery load job needs, decoupled from the client for testability.
Captures what a class logs, so a test can assert the line rather than only its side effects.
One collected log event, flattened out of log4j2's own type.
The levels a capture can be taken at, so no log4j2 type reaches this class's signature.
Publishes every container's ports on the loopback address the Docker host resolves to, rather than on the wildcard address, so a host-port collision with an unrelated local process cannot form (ADR-0132).
A materialised Bigtable row and the stable byte estimate measured while it was decoded.
Materialises SDK rows while measuring their decoded wire content without traversing them twice.
Shared boundary handling for connector metric values.
Fails a reader that holds unacknowledged messages but has never been asked for a checkpoint.
Checkpointed timer for a service partition absent from the coordinator ledger.
JSON values describing the changes to one watched table row.
The operation represented by one Spanner change-stream modification.
Accepts row mutations for one table and reports each one's outcome separately.
Creates one MutationBatcher per table the writer resolves, and owns whatever those batchers share.
Appends row batches to one buffered write stream at explicit offsets.
The checks every connector's option builders repeat, in one place.
Assertions shared by the Table API option-set tests.
Applies Table API option values to DataStream builder setters, renaming a rejected value's failure to the option key the SQL caller actually wrote (issue #1030, ADR-0133).
Applies Table API option values to DataStream builder setters, renaming a rejected value's failure to the option key the SQL caller actually wrote (issue #1030, ADR-0133).
Applies Table API option values to DataStream builder setters, renaming a rejected value's failure to the option key the SQL caller actually wrote (issue #1030, ADR-0133).
Applies Table API option values to DataStream builder setters, renaming a rejected value's failure to the option key the SQL caller actually wrote (issue #1030, ADR-0133).
Applies Table API option values to DataStream builder setters, renaming a rejected value's failure to the option key the SQL caller actually wrote (issue #1030, ADR-0133).
Whether the source preserves Pub/Sub ordering-key delivery order.
How StagingFormat.PARQUET staging files are compressed.
Reports successful end-of-query after all child-partitions records were forwarded.
Lifecycle of one Spanner Change Streams partition in the checkpointed coordinator ledger.
What one planning call answered with: a snapshot, and the partitions of the read at it.
Opens a batch read and asks the service to plan it into partitions.
Mints the PartitionPlanner one enumerator plans through.
Reader heartbeat progress used to keep the coordinator's recovery watermark current.
Reader progress used to keep the coordinator's assigned-partition ledger current.
Reports that a partition closed and names each successor range and its continuation token.
One target range from CloseStream paired with the token that arrived from this parent.
How much a paused split may buffer before the reader stops its subscriber.
Successor partition parked until its parent tokens cover the whole target range.
A split as the planner produced it, with the size the samples suggest it holds.
Adds fields to a protobuf row descriptor without changing its surrounding file structure.
ProtoMessageSerializationSchema<T extends com.google.protobuf.Message>
A BigQueryProtoSerializationSchema for records that already are protobuf messages.
One validated field consumed by the generic protobuf row-augmentation engine.
Serializable provider whose validation may report a row-level IOException.
Write-only fields accepted by BigQuery's documented Storage Write API contract.
Adds ordered physical or write-only fields to delegate protobuf rows.
Rewrites protobuf messages into DynamicMessages conforming to a BigQuery-storage compatible target descriptor (as produced by BQTableSchemaToProtoDescriptor from the schema derived by ProtoToTableSchemaConverter).
Options controlling how protobuf descriptors are mapped to BigQuery schemas.
Builder for ProtoSchemaOptions.
Converts serialized proto rows (the BigQueryProtoSerializationSchema wire form, whose descriptors follow the BQTableSchemaToProtoDescriptor conventions) into Avro GenericRecords conforming to the schema produced by TableSchemaToAvroConverter.
Derives a BigQuery TableSchema from a protobuf Descriptors.Descriptor.
Creates TopicPublisher instances for the writer's per-topic publisher map.
Builds PubSubPublisherOptions from the table options.
AckTracker keeping the acknowledgement handles in memory until their checkpoint completes.
The WITH options of the pubsub table connector.
Loads the credentials shared by the Pub/Sub source and sink client families.
A DeadLetterQueue publishing every terminally failed element to a Pub/Sub topic, used through FailureHandler.sendToDeadLetterQueue(DeadLetterQueue).
Deserializes Pub/Sub messages into source records.
A table sink publishing rows to one Pub/Sub topic.
A table source consuming rows from one or more Pub/Sub subscriptions.
The pubsub table connector factory.
The Pub/Sub emulator image shared by every harness that starts the emulator, so they cannot drift apart.
Checkpointed state of the split enumerator: the subscriptions it resolved.
Serializer for PubSubEnumeratorState.
Every metric name this connector registers itself, in one place so that this file is the connector's inventory: what it reports can be read here without opening a writer, a reader or the enumerator.
Tuning options for the sink's Pub/Sub publishers and its writer: SDK batching and publish-retry settings, message ordering, the writer's in-flight caps, the backoff budget of the topic auto-creation recovery, and the writer's shutdown budget.
At-least-once sink publishing through google-cloud-pubsub Publisher instances with dynamic per-record topic destinations.
Deserializes received messages and emits them, using the Pub/Sub publish time as the event timestamp.
Serializes sink records into Pub/Sub messages.
Extracts message attributes from a record.
Extracts the ordering key from a record.
What this connector's bounded teardowns leave behind, counted for the lifetime of the class loader rather than of a task.
Entry point for building a Pub/Sub sink.
Builder for Pub/Sub sinks, obtained from PubSubSink.builder().
Immutable sink configuration assembled by PubSubSinkBuilder.
Entry point for building Pub/Sub sources.
Builder for Pub/Sub sources, obtained from PubSubSource.builder().
Immutable configuration of a Pub/Sub source, assembled by PubSubSourceBuilder and shipped in the job graph.
Reader subtask consuming its assigned subscription splits.
The reader's Pub/Sub-specific metrics.
Verifies the subscriptions, then assigns subscription splits to reader subtasks.
Multiplexes the streaming-pull subscribers of one reader subtask's splits.
Fetch-loop helpers for driving a Pub/Sub SplitReader directly in tests, typed against the flink-connector-base interface so the connector's reader class never crosses the module boundary.
Where the source starts consuming a subscription.
Which starting point a PubSubStartPosition names.
At-least-once source consuming Pub/Sub subscriptions through google-cloud-pubsub Subscriber streaming pull.
Tuning options for the source's Pub/Sub subscribers and its reader: SDK flow control, the streaming-pull connection count and the acknowledgement-deadline extension settings, plus the source's own drain size, subscriber shutdown budget and first-checkpoint watchdog.
Default SubscriptionAdmin backed by the Pub/Sub SubscriptionAdminClient.
The Pub/Sub admin, publish and pull machinery every integration-test harness needs, parameterised over the transport: a plaintext channel with no credentials against the emulator, or application-default credentials against the real service.
Default TopicAdmin backed by the Pub/Sub TopicAdminClient.
At-least-once writer publishing records to dynamic per-record Pub/Sub topic destinations.
PullAssignmentSplitEnumerator<SplitT extends org.apache.flink.api.connector.source.SourceSplit,StateT,PlanT>
A bounded source's split enumerator that plans once, asynchronously, and then hands the plan out one split per request.
Bridges Pub/Sub's push-style streaming pull to the pull-style SplitReader contract: the client library delivers messages on its own threads into an in-memory buffer, and the fetcher thread drains that buffer.
The deterministic identity under which a query job may be reused across a re-plan.
Everything one terminal BigQuery query job needs, decoupled from the client for testability.
What running the query produced: the table the result landed in, and whether it came from a previous attempt's job rather than a fresh submission.
Runs the source's query and says which table its result landed in.
The query job a QueryRunner is asked to run.
A fully-qualified Cloud Tasks queue reference: project, location and queue.
Opens ReadRows calls through a BigQueryReadClient.
Creates read sessions through a BigQueryReadClient.
Absolute number of additional partition splits one reader can accept.
Adapts a SourceReaderContext to the context a DeserializationSchema expects when a source opens it.
Creates the source's read session.
Mints the ReadSessionCreator one enumerator plans through.
One Storage Read API ReadStream, plus how far into it the source has already consumed.
Serializer for ReadStreamSplit.
The mutable reading progress of one ReadStreamSplit.
A read-only Counter view of one PubSubShutdownResidue adder, so a residue registers as the counter it is rather than as a gauge over a monotonic total.
The check a configured value earns by being a component of a resource path this project composes.
A TableAdmin failure that repeating the same call can fix — a rate limit, a quota that refills, a server-side error.
Support for the connectors' hand-written retry loops.
A TableAdmin that repeats a creation the service answered with a RetriableTableAdminException, within a fixed budget.
An immutable retry schedule: exponential backoff from an initial delay up to a cap, a bounded number of attempts, and optional proportional jitter (to de-synchronize parallel subtasks retrying against the same destination).
Appends batches of serialized rows to one destination's default write stream.
Creates RowAppenders for destinations.
How RowTypeToTableSchemaConverter derives a BigQuery schema from a SQL table's columns — the RowData counterpart of AvroSchemaOptions, carrying the same three knobs under the same names.
Serializes table-runtime rows into mutations for one configured table.
Derives the protobuf row descriptor a BigQuery table schema is written through.
How a Bigtable table option represents a row-key bound.
One row key the service offered as a section boundary, with the approximate number of bytes stored before it.
Asks the service where a table's sections begin.
Mints the RowKeySampler one enumerator plans against.
The module's row-key range algebra: emptiness, containment, cutting, coalescing, intersection, truncation and the unsigned key comparison they all rest on, each defined once here.
One contiguous row-key range still to be read.
Cuts the configured row-key ranges at the boundaries the service sampled.
Serializer for RowRangeSplit.
How far through one RowRangeSplit the source has emitted.
One open ReadRows call: the response blocks of one stream, from one offset onwards.
One open ReadRows call, consumed a row at a time.
Opens ReadRows calls on a read session's streams.
Opens a ReadRows call over one row-key range.
Which Bigtable read surface a table source uses.
The Spanner read path selected by a table DDL.
Computes the union of a destination table's live schema and the serializer's desired schema — the schema the sink proposes when connector-driven schema updates are enabled.
A checked rejection of a schema union: the desired schema requires a change the rules (or the configured options) do not permit.
The result of a union: the unified schema and whether it differs from the existing one.
Options gating connector-driven destination table schema updates.
Builder for SchemaUpdateOptions.
The physical primary key and value-format payload of a selected-cell table.
Creates synthetic service-account keys for credential wiring tests: structurally valid enough for the Google auth libraries to parse — a real RSA key in a real PEM envelope — while naming an example.invalid account no service would ever accept, so a test that leaks one at a real endpoint fails loudly rather than authenticating.
Locates the uber-jar a flink-sql-connector-gcp-* module builds, for the tests that inspect or run against it.
What the split enumerator checkpoints: whether the read has been planned, and the splits nobody is holding yet.
The bounded batch source over Spanner's partitioned reads.
Opens one batch read, asks Spanner to plan it into partitions, and hands the partitions out one at a time.
Builds BigQuery CDC sequence numbers from typed Spanner change-stream coordinates.
Transcodes Spanner change-stream coordinates into BigQuery CDC hexadecimal sections.
Metadata operations the Spanner Change Streams coordinator needs during initialization.
Serializable job-graph seam that opens the coordinator's runtime metadata client.
Turns one Spanner data-change record into zero or more user records.
An unbounded Table API source for one table observed through a Spanner change stream.
Checkpointed Spanner Change Streams partition state.
Connector-owned checkpoint format for the Spanner Change Streams partition ledger.
Releases readers after the coordinator has validated Change Stream metadata and restore age.
Opens asynchronous partition queries for one source subtask.
 
 
Serializable factory for the TaskManager-local Change Streams query client.
Reads several Spanner Change Streams partition queries concurrently in one source subtask.
Applies connector-side table and column filters to decoded data-change records.
Result of applying table and column filters to one data-change record.
 
FLIP-27 source for Cloud Spanner Change Streams.
Immutable configuration assembled by SpannerChangeStreamSourceBuilder.
Coordinates the checkpointed parent-child lifecycle of Spanner Change Streams partitions.
Advances every reader subtask to the coordinator's unfinished-ledger watermark.
Calculates the non-early Flink watermark for the complete Spanner partition ledger.
Builds the Spanner service handle both directions of this connector open.
The WITH options of the spanner table connector.
Loads credentials shared by every direction of this connector: the sink writer, the bounded source's planner and readers, the Change Streams coordinator and query clients, and the table lookup path.
The narrow view of a Spanner database the writer needs: read the schema once, then apply batches.
Receives the outcome of one mutation group of a batch write.
Opens a SpannerDatabaseAccess on the task manager.
The spanner table sink, backed by the DataStream SpannerSink.
Collects values for the immutable sink.
A bounded table scan and primary-key point-lookup source for Spanner tables.
Creates the spanner table source and sink from a SQL DDL.
The Spanner emulator image shared by every harness that starts the emulator, so they cannot drift apart.
Lookup options owned by the Spanner table source.
Every metric name this connector registers, in one place.
Turns a stream record into the Spanner Mutation the sink writes.
At-least-once sink applying one mutation per record through batchWriteAtLeastOnce of google-cloud-spanner.
What the batch source reads: either a query, or a table with its columns and key set, optionally through an index.
Internal bridge between a table-layer deferred read and the batch partition planner.
Resolves a table-layer read after the batch transaction has selected its snapshot.
Deserializes each row and hands its output records on.
Asynchronous primary-key lookup producing the projected table row.
Synchronous primary-key lookup producing the projected table row.
The priority Spanner schedules this connector's requests at.
Entry point for building a Spanner sink.
Builder for the Spanner sink; created through SpannerSink.builder().
The immutable configuration SpannerSinkBuilder builds and the sink carries to the task managers.
Reads rows from a Cloud Spanner database.
Builds a SpannerSource.
Everything the batch source was built with, assembled by the builder and carried into the job graph.
Reads the partitions this subtask is assigned, asking the enumerator for the next one each time it finishes one.
The batch reader's own counters, beside the ones Flink registers for every source.
Reads the partitions this subtask was assigned, a bounded batch at a time.
Turns a Spanner row into zero or more output records.
A table name as rendered for Spanner APIs and compared with INFORMATION_SCHEMA.
The name of one table-local access path, such as an index.
The physical DDL schema and its corresponding native Spanner types.
One physical column.
Stock Spanner clients pointed at an emulator endpoint.
The Spanner sink's writer: buffers mutations, applies them with batchWriteAtLeastOnce, and decides per mutation what a refusal means.
Tuning options for the sink's writer: how large a batch write request grows, how Spanner should schedule it, and the recovery budget the writer spends on transient failures.
The split universe of a job and its subtask ownership, computed deterministically from the subscription list, the ordering mode and the source parallelism.
The file format FILE_LOADS stages rows in before loading them, set by FileLoadsOptions.Builder#stagingFormat.
The Cloud Storage staging area of the FILE_LOADS write method.
Where a change-stream source starts reading when no checkpointed state is restored.
Builds a PubSubStartPosition from the table options.
Resolves change-stream start positions against one startup instant and one retention window.
The retained-window evidence for one expired restored partition.
Discovers how long a connector's change stream retains records.
Extraction of gRPC status codes from the exceptions the Google Cloud clients surface.
Converts a Storage API TableSchema (the schema form of the serializer SPI) into the REST client Schema needed to create tables.
Default RowAppenderFactory backed by Storage Write API StreamWriters on the destination's default write stream.
One open read of a partition, consumed a row at a time.
Rejoins a batch read's snapshot and opens one of its partitions.
The parts of WriterInitContext a sink reads when creating a writer; everything a sink has no reason to touch is unsupported, so a new dependency on the context shows up as a failing test rather than as a silent null.
Reports that one source reader could not admit a delivery within its subscriber-buffer bound.
Creates the google-cloud-pubsub Subscriber backing one split.
Receives a message together with the handle that settles it.
Builds PubSubSubscriberOptions from the table options.
Subscription administration used by the enumerator's startup check, abstracting the Pub/Sub admin client so the check can be unit-tested without one.
Settings the source applies when it creates a subscription that does not exist.
Builds per-subscription SubscriptionCreateOptions from the table options.
A fully-qualified Pub/Sub subscription reference: project and subscription.
The settings of an existing subscription that the source's preflight reads.
Builder for SubscriptionInfo.
A unit of work for one reader subtask: a streaming-pull connection to one subscription.
Serializer for SubscriptionSplit.
Runs one source deserialization call with a direct, non-null collector.
One deserialization call that emits through the supplied collector.
Administers destination tables: creation for CreateDisposition.CREATE_IF_NEEDED and schema reads/updates for schema evolution.
Table administration operations used by the sink, abstracting the Bigtable admin client so writer logic can be unit-tested without one.
What an TableAdmin.ensureTable(io.github.flink.gcp.connector.bigtable.TableDestination, io.github.flink.gcp.connector.bigtable.sink.TableCreateOptions) call actually did, so the caller can report creation and family-addition separately without this SPI knowing about metrics.
A table-administration failure that retains whether the operation requested creation.
Options applied when the sink creates a destination table under CreateDisposition.CREATE_IF_NEEDED: time partitioning and clustering.
The settings for the table the sink creates under CreateDisposition.CREATE_IF_NEEDED: its column families and, per family, an optional garbage-collection rule.
Builder for TableCreateOptions.
Builder for TableCreateOptions.
Granularity of time-based partitioning.
Maps the sink.table-create.* options onto TableCreateOptions.
Builds TableCreateOptions from the table options and the DDL's column families.
Supplies per-destination TableCreateOptions for tables created under CreateDisposition.CREATE_IF_NEEDED.
A fully-qualified BigQuery table reference: project, dataset and table.
A fully-qualified Bigtable table reference: project, instance and table.
A point-in-time read of a destination table's schema, as returned by TableAdmin.getSchema(io.github.flink.gcp.connector.bigquery.sink.TableDestination).
Converts a Storage API TableSchema (the schema form of the serializer SPI) into the Avro schema staged files are written with.
Validated fixed target values used by the table sink.
Creates Cloud Tasks tasks, one RPC per task.
Creates the writer's TaskCreator.
Extracts the deduplication key of a record, opting the sink into named tasks.
Shared test stubs for SinkWriter.Context.
Random resource-name helpers for tests whose resources share a namespace — the gated real-GCP suites above all, where concurrent runs must not collide on resource names and a crashed run must leave behind names that identify it, but also an emulator harness giving each test a database of its own.
A SinkCommitterMetricGroup whose metrics can be read back by the names they registered under — TestSinkCommitterMetricGroup is to a committer what TestSinkWriterMetricGroup is to a writer, and for the same reason: UnregisteredMetricsGroup.createSinkCommitterMetricGroup() hands out a fresh SimpleCounter per call, so a counter the committer registered is unreachable afterwards.
A SinkWriterMetricGroup whose metrics can be read back by the names they registered under, so a sink writer's metrics are asserted the way a reporter would see them.
Transcodes a TiCDC commit TSO into a BigQuery CDC sequence section.
Derives BigQuery CDC sequence numbers from TiCDC's Debezium-protocol source properties.
The writer's view of time: reading the clock and waiting out a retry backoff.
Topic administration operations used by the sink, abstracting the Pub/Sub admin client so writer logic can be unit-tested without one.
Settings the sink applies when it creates a topic that does not exist.
Builder for TopicCreateOptions.
Builds TopicCreateOptions from the table options.
A fully-qualified Pub/Sub topic reference: project and topic.
Publishes messages to one Pub/Sub topic.
What a fixed-width decoder does with a cell or row key longer than the declared type's layout.
An explicit, record-specific destination-resolution failure.
Parses the table layer's string carrier as a native Spanner UUID value.
The value-capture policy that was active when a data change was recorded.
Production BufferedStreamService over one BigQueryWriteClient.
How data lands in a destination table that already contains data.
The mechanism used to write records to BigQuery.
Builds BigtableWriterOptions from the table options.
Maps Table API options onto the DataStream writer options.
Maps table options onto SpannerWriterOptions.