Package io.github.flink.gcp.connector.cloudtasks.sink

Interface DestinationResolver<T>

Type Parameters:
T - type of the records written by the sink
All Superinterfaces:
Serializable
All Known Implementing Classes:
FixedDestinationResolver
Functional Interface:
This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@Public @FunctionalInterface public interface DestinationResolver<T> extends Serializable
Resolves the destination queue for each record, enabling one sink instance to write to many queues (dynamic destinations).

Unlike the Pub/Sub and BigQuery sinks' dynamic destinations this costs the sink nothing: Cloud Tasks has no per-destination connection or stream, so one client serves every queue. Sharding across queues is also the way to raise aggregate throughput past a single queue's limits.

The resolver is invoked once per record on the hot write path. Implementations must be deterministic and cheap; when destinations repeat, return cached QueueDestination instances instead of re-creating them per record (for example via a small Map#computeIfAbsent keyed on the varying component).

The SinkWriter.Context exposes the record's event timestamp for time-based routing.

  • Method Summary

    Modifier and Type
    Method
    Description
    QueueDestination
    resolve(T element, org.apache.flink.api.connector.sink2.SinkWriter.Context context)
    Returns the destination queue for the given record.

  • Method Details

    • resolve

      QueueDestination resolve(T element, org.apache.flink.api.connector.sink2.SinkWriter.Context context)
      Returns the destination queue for the given record.
      Parameters:
      element - the record
      context - writer context exposing the record's event timestamp and current watermark
      Returns:
      the destination queue; never null