Package io.github.flink.gcp.connector.bigtable.sink

Class BigtableSink

java.lang.Object
io.github.flink.gcp.connector.bigtable.sink.BigtableSink
@Public public final class BigtableSink extends Object
Entry point for building a Bigtable sink.

The sink applies one row mutation per record through the client's bulk MutateRows batcher, at-least-once, and waits for every outstanding mutation at each checkpoint barrier. A replayed record overwrites the same cells only when the serializer sets explicit cell timestamps; see BigtableSerializationSchema.

That at-least-once statement assumes the default FailureHandler.failJob() policy. Under a dropping policy configured through BigtableSinkBuilder.failedMutationHandler(FailureHandler), a completed checkpoint means every record up to the barrier was either applied, skipped by the serializer, or handed to that handler.

The sink writes to one fixed table, or to a table it resolves per record: BigtableSinkBuilder.table(TableDestination) names one, and BigtableSinkBuilder.destinationResolver(DestinationResolver) routes each record to a table of its own. By default it never creates a table — every table it writes to, and its column families, must exist; BigtableSinkBuilder.createDisposition(CreateDisposition) with CreateDisposition.CREATE_IF_NEEDED and BigtableSinkBuilder.tableCreateOptions(TableCreateOptions) opts into creating them, from one schema that serves every table the sink creates.

Example: 


 Sink<OrderEvent> sink =
         BigtableSink.<OrderEvent>builder()
                 .table(TableDestination.of("my-project", "my-instance", "orders"))
                 .serializer(
                         (event, context) ->
                                 RowMutationEntry.create(event.getId())
                                         .setCell(
                                                 "cf",
                                                 "payload",
                                                 event.getTimestampMicros(),
                                                 event.getBody()))
                 .build();

  • Method Details