Package io.github.flink.gcp.connector.spanner.sink

Class SpannerWriterOptions

java.lang.Object
io.github.flink.gcp.connector.spanner.sink.SpannerWriterOptions
All Implemented Interfaces:
Serializable
@Public public final class SpannerWriterOptions extends Object implements Serializable
Tuning options for the sink's writer: how large a batch write request grows, how Spanner should schedule it, and the recovery budget the writer spends on transient failures.

Set via SpannerSinkBuilder.writerOptions(SpannerWriterOptions); optional — every knob is defaulted, so defaults() is equivalent to not setting options at all.

What the batch limits defend

A sink that accumulates mutations has to bound the request it builds, since a request Spanner refuses is refused as a whole, taking every mutation in it with it. That much is correctness rather than tidiness. Which limit each knob defends is narrower than three of them make it look: Spanner documents no per-request mutation count for batch write at all, so SpannerWriterOptions.Builder.maxBatchBytes(long) is the one defending a documented request-level limit, and the other two bound the request as a proxy for its size. How large that size limit is can be read two ways, and the ceiling below takes the looser one. docs/adr/0077 carries the documentation rows this rests on.

SpannerWriterOptions.Builder.maxBatchCells(int) is counted the way Spanner counts a mutation: a written column costs one cell for the table plus one for every secondary index containing it, read from the database's INFORMATION_SCHEMA when the writer opens. On a wide row that is a better proxy for the request's size than a count of mutations would be.

The defaults are Apache Beam's, and Beam batches for Commit rather than for batch write — which is where the commit-shaped figures entered this connector. They sit far under every reading of every limit all the same.

Why there are recovery knobs at all

Unlike every other Google client this project builds on, the Spanner client does not retry the RPC this sink writes with: SpannerStubSettings configures batchWrite with an empty retryable-code set (checked against google-cloud-spanner 6.120.0), and the only retry around it re-creates a lost session. So the sink owns the whole retry loop — the Cloud Tasks shape rather than the Bigtable one — and these knobs budget it.

Instances are immutable and serializable.

See Also:

  • Field Details

  • Method Details

    • builder

      public static SpannerWriterOptions.Builder builder()
      Creates a new SpannerWriterOptions.Builder.
      Returns:
      a new builder

    • defaults

      public static SpannerWriterOptions defaults()
      Returns the default options: at most 5000 mutation cells, 500 mutations and 1 MiB per batch write request, the service's own commit delay and priority, a 30 s timeout for one complete batch write attempt, and a recovery budget of 500 ms doubling to 10 s over at most 10 attempts.
      Returns:
      the default options

    • getMaxBatchCells

      public int getMaxBatchCells()
      Returns the cap on mutation cells per batch write request, index entries included.

    • getMaxBatchMutations

      public int getMaxBatchMutations()
      Returns the cap on mutations per batch write request.

    • getMaxBatchBytes

      public long getMaxBatchBytes()
      Returns the cap on the estimated size of a batch write request.

    • getMaxCommitDelay

      @Nullable public Duration getMaxCommitDelay()
      Returns the commit delay, or null to leave the service's own handling in place.

    • getRpcPriority

      @Nullable public SpannerRpcPriority getRpcPriority()
      Returns the RPC priority, or null to leave it unspecified (meaning high).

    • getBatchWriteTimeout

      public Duration getBatchWriteTimeout()
      Returns the timeout for one complete batch write attempt.

    • getRecoveryInitialBackoff

      public Duration getRecoveryInitialBackoff()
      Returns the first backoff of the writer's retry loop.

    • getRecoveryMaxBackoff

      public Duration getRecoveryMaxBackoff()
      Returns the backoff cap of the writer's retry loop.

    • getRecoveryMaxAttempts

      public int getRecoveryMaxAttempts()
      Returns the maximum attempts of the writer's retry loop.

    • toRecoverySchedule

      @Internal public RetrySchedule toRecoverySchedule()
      Returns the retry schedule the recovery* knobs describe. Jittered: every subtask writing to a database that has just become unavailable retries on the same schedule, so unjittered they would all come back at the same instant.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object