Package io.github.flink.gcp.connector.cloudtasks.sink

Class CloudTasksSinkBuilder<T>

java.lang.Object
io.github.flink.gcp.connector.cloudtasks.sink.CloudTasksSinkBuilder<T>
Type Parameters:
T - type of the records written by the sink
@Public public class CloudTasksSinkBuilder<T> extends Object
Builder for Cloud Tasks sinks, obtained from CloudTasksSink.builder().

Required settings: a serialization schema and a destination. The destination is set through either queue(QueueDestination) (fixed queue) or destinationResolver(DestinationResolver) (per-record dynamic destinations); the two override each other and the last call wins.

The queue itself is never created by the sink and must exist: an auto-created queue would carry Cloud Tasks' default rate limits, silently discarding the pacing that is the reason to use the service. A deleted queue name can also remain temporarily unavailable for reuse.

  • Method Details

    • queue

      public CloudTasksSinkBuilder<T> queue(QueueDestination queue)
      Creates every task in the given fixed queue. Overrides any previously set queue or resolver.
      Parameters:
      queue - the destination queue
      Returns:
      this builder

    • destinationResolver

      public CloudTasksSinkBuilder<T> destinationResolver(DestinationResolver<? super T> destinationResolver)
      Resolves the destination queue per record (dynamic destinations). Overrides any previously set queue or resolver.
      Parameters:
      destinationResolver - the resolver
      Returns:
      this builder

    • serializer

      public CloudTasksSinkBuilder<T> serializer(CloudTasksSerializationSchema<? super T> serializer)
      Sets the record serialization schema. Required; a schema returning null skips the record rather than failing it.
      Parameters:
      serializer - the serialization schema
      Returns:
      this builder

    • taskIdExtractor

      public CloudTasksSinkBuilder<T> taskIdExtractor(TaskIdExtractor<? super T> taskIdExtractor)
      Deduplicates records by the extracted key. Optional; without it the default at-least-once mode creates unnamed tasks, while exactly-once mode persists a fresh random identity for every accepted record. Recovery in exactly-once mode reuses the saved identity.

      The sink hashes the key with SHA-256 before using it as the task id, and a repeated create for a key Cloud Tasks still remembers counts as success. Naming is off by default because Google documents the duplicate-name lookup as significantly increasing create latency, and because the window in which a key is remembered is bounded — its own documentation gives both "up to 24 hours" and "~1 hour" for it. Neither estimate establishes a precise minimum retention period for a correctness deadline. The sink does not verify queue retention or enforce a bounded recovery protocol.

      Parameters:
      taskIdExtractor - the deduplication-key extractor
      Returns:
      this builder

    • writerOptions

      public CloudTasksSinkBuilder<T> writerOptions(CloudTasksWriterOptions writerOptions)
      Sets the in-flight cap and the two retry budgets for the writer or staged committer. Optional; defaults to CloudTasksWriterOptions.defaults().
      Parameters:
      writerOptions - the options
      Returns:
      this builder

    • failedTaskHandler

      public CloudTasksSinkBuilder<T> failedTaskHandler(FailureHandler<? super FailedTask> failedTaskHandler)
      Sets the policy for a task that terminally fails — fail the job (the default), drop it, or send it to a dead-letter queue. Only data-shaped failures reach it: a record the serializer rejects, a task id extractor that throws, and a creation the service rejects with INVALID_ARGUMENT. Everything else keeps failing the job, including an exhausted retry budget and PERMISSION_DENIED — see the connector documentation for the full routing table.

      The parameter is contravariant, so a handler written against the shared FailedElement contract serves every connector in this repository without a cast.

      Parameters:
      failedTaskHandler - the handler
      Returns:
      this builder

    • serviceAccountKeyFile

      public CloudTasksSinkBuilder<T> serviceAccountKeyFile(String serviceAccountKeyFile)
      Authenticates the sink with the service-account JSON key at the given path instead of application-default credentials. The file is read on each TaskManager when its writer or committer is created, so the same path must be readable by every TaskManager that can run this sink. Optional; when unset the sink uses application-default credentials.

      Service-account keys are long-lived secrets. Prefer an attached service account or Workload Identity where the deployment supports one. This setting cannot be combined with emulatorEndpoint(String), whose plaintext channel deliberately carries no credentials.

      Parameters:
      serviceAccountKeyFile - the service-account JSON key-file path
      Returns:
      this builder

    • emulatorEndpoint

      public CloudTasksSinkBuilder<T> emulatorEndpoint(String emulatorEndpoint)
      Points the sink at a Cloud Tasks emulator instead of the production service. The connection to the given host:port uses a plaintext channel with no credentials, so this must only ever be used against an emulator. Optional; when unset the sink connects to Cloud Tasks with application-default credentials.

      The value is parsed here, so a malformed host:port is rejected on the client instead of surfacing as a connection failure once the job has been deployed.

      Parameters:
      emulatorEndpoint - the emulator endpoint as host:port
      Returns:
      this builder
      Throws:
      IllegalArgumentException - if the endpoint is not host:port with a port in 1..65535

    • deliveryGuarantee

      public CloudTasksSinkBuilder<T> deliveryGuarantee(CloudTasksDeliveryGuarantee deliveryGuarantee)
      Selects eager creation or checkpointed named creation within the documented recovery window.
      Parameters:
      deliveryGuarantee - the task-creation mode
      Returns:
      this builder

    • stagedOptions

      public CloudTasksSinkBuilder<T> stagedOptions(CloudTasksStagedOptions stagedOptions)
      Sets staging and recovery options; requires CloudTasksDeliveryGuarantee.EXACTLY_ONCE.
      Parameters:
      stagedOptions - the staging and recovery options
      Returns:
      this builder

    • build

      public org.apache.flink.api.connector.sink2.Sink<T> build()
      Builds the sink.
      Returns:
      the sink