Package io.github.flink.gcp.connector.cloudtasks.sink

Class CloudTasksSinkBuilder<T>

java.lang.Object
io.github.flink.gcp.connector.cloudtasks.sink.CloudTasksSinkBuilder<T>
Type Parameters:
T - type of the records written by the sink
@Public public class CloudTasksSinkBuilder<T> extends Object
Builder for Cloud Tasks sinks, obtained from CloudTasksSink.builder().

Required settings: a serialization schema and a destination. The destination is set through either queue(QueueDestination) (fixed queue) or destinationResolver(DestinationResolver) (per-record dynamic destinations); the two override each other and the last call wins.

The queue itself is never created by the sink and must exist: an auto-created queue would carry Cloud Tasks' default rate limits, silently discarding the pacing that is the reason to use the service, and a deleted queue name cannot be reused for 3 days.

  • Method Details

    • queue

      public CloudTasksSinkBuilder<T> queue(QueueDestination queue)
      Creates every task in the given fixed queue. Overrides any previously set queue or resolver.
      Parameters:
      queue - the destination queue
      Returns:
      this builder

    • destinationResolver

      public CloudTasksSinkBuilder<T> destinationResolver(DestinationResolver<? super T> destinationResolver)
      Resolves the destination queue per record (dynamic destinations). Overrides any previously set queue or resolver.
      Parameters:
      destinationResolver - the resolver
      Returns:
      this builder

    • serializer

      public CloudTasksSinkBuilder<T> serializer(CloudTasksSerializationSchema<? super T> serializer)
      Sets the record serialization schema. Required; a schema returning null skips the record rather than failing it.
      Parameters:
      serializer - the serialization schema
      Returns:
      this builder

    • taskIdExtractor

      public CloudTasksSinkBuilder<T> taskIdExtractor(TaskIdExtractor<? super T> taskIdExtractor)
      Opts into named tasks, deduplicating records by the extracted key. Optional; without it the sink creates unnamed tasks and a record replayed after a failure calls the endpoint twice.

      The sink hashes the key with SHA-256 before using it as the task id, and a repeated create for a key Cloud Tasks still remembers counts as success. Naming is off by default because Google documents the duplicate-name lookup as significantly increasing create latency, and because the window in which a key is remembered is bounded — its own documentation gives both "up to 24 hours" and "~1 hour" for it, so design against the shorter one.

      Parameters:
      taskIdExtractor - the deduplication-key extractor
      Returns:
      this builder

    • writerOptions

      public CloudTasksSinkBuilder<T> writerOptions(CloudTasksWriterOptions writerOptions)
      Sets the writer tuning options (the in-flight cap and the two retry budgets). Optional; defaults to CloudTasksWriterOptions.defaults().
      Parameters:
      writerOptions - the options
      Returns:
      this builder

    • failedTaskHandler

      public CloudTasksSinkBuilder<T> failedTaskHandler(FailureHandler<? super FailedTask> failedTaskHandler)
      Sets the policy for a task that terminally fails — fail the job (the default), drop it, or send it to a dead-letter queue. Only data-shaped failures reach it: a record the serializer rejects, a task id extractor that throws, and a creation the service rejects with INVALID_ARGUMENT. Everything else keeps failing the job, including an exhausted retry budget and PERMISSION_DENIED — see the connector documentation for the full routing table.

      The parameter is contravariant, so a handler written against the shared FailedElement contract serves every connector in this repository without a cast.

      Parameters:
      failedTaskHandler - the handler
      Returns:
      this builder

    • serviceAccountKeyFile

      public CloudTasksSinkBuilder<T> serviceAccountKeyFile(String serviceAccountKeyFile)
      Authenticates the sink with the service-account JSON key at the given path instead of application-default credentials. The file is read on each TaskManager when its writer is created, so the same path must be readable by every TaskManager that can run this sink. Optional; when unset the sink uses application-default credentials.

      Service-account keys are long-lived secrets. Prefer an attached service account or Workload Identity where the deployment supports one. This setting cannot be combined with emulatorEndpoint(String), whose plaintext channel deliberately carries no credentials.

      Parameters:
      serviceAccountKeyFile - the service-account JSON key-file path
      Returns:
      this builder

    • emulatorEndpoint

      public CloudTasksSinkBuilder<T> emulatorEndpoint(String emulatorEndpoint)
      Points the sink at a Cloud Tasks emulator instead of the production service. The connection to the given host:port uses a plaintext channel with no credentials, so this must only ever be used against an emulator. Optional; when unset the sink connects to Cloud Tasks with application-default credentials.

      The value is parsed here, so a malformed host:port is rejected on the client instead of surfacing as a connection failure once the job has been deployed.

      Parameters:
      emulatorEndpoint - the emulator endpoint as host:port
      Returns:
      this builder
      Throws:
      IllegalArgumentException - if the endpoint is not host:port with a port in 1..65535

    • build

      public org.apache.flink.api.connector.sink2.Sink<T> build()
      Builds the sink.
      Returns:
      the sink