Package io.github.flink.gcp.connector.bigtable.sink

Interface DestinationResolver<T>

Type Parameters:
T - type of the records written by the sink
All Superinterfaces:
Serializable
All Known Implementing Classes:
FixedDestinationResolver
Functional Interface:
This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.
@Public @FunctionalInterface public interface DestinationResolver<T> extends Serializable
Resolves the destination table for each record, enabling one sink instance to write to many tables (dynamic destinations).

The resolver is invoked once per record on the hot write path, and before the serializer runs — a record the serializer then rejects is reported against the table it was headed for. Implementations must be deterministic and cheap; when destinations repeat, return cached TableDestination instances instead of re-creating them per record (for example via a small Map#computeIfAbsent keyed on the varying component).

Each distinct table costs the writer a bulk mutation batcher of its own, held until the destination goes idle, so a resolver's cardinality is a sink resource decision and not only a routing one. BigtableWriterOptions.destinationIdleTimeout is what bounds it.

The SinkWriter.Context exposes the record's event timestamp for time-based routing.

  • Method Summary

    Modifier and Type
    Method
    Description
    TableDestination
    resolve(T element, org.apache.flink.api.connector.sink2.SinkWriter.Context context)
    Returns the destination table for the given record.

  • Method Details

    • resolve

      TableDestination resolve(T element, org.apache.flink.api.connector.sink2.SinkWriter.Context context)
      Returns the destination table for the given record.
      Parameters:
      element - the record
      context - writer context exposing the record's event timestamp and current watermark
      Returns:
      the destination table; never null