Package io.github.flink.gcp.connector.spanner.sink

Class SpannerWriterOptions.Builder

java.lang.Object
io.github.flink.gcp.connector.spanner.sink.SpannerWriterOptions.Builder
Enclosing class:
SpannerWriterOptions
@Public public static final class SpannerWriterOptions.Builder extends Object
Builder for SpannerWriterOptions.

  • Method Details

    • maxBatchCells

      public SpannerWriterOptions.Builder maxBatchCells(int maxBatchCells)
      Caps the mutation cells the writer puts in one batch write request. Defaults to 5000.

      Cells are counted as Spanner counts a mutation: an insert or update costs one cell per column it writes — primary-key columns always count — plus one for every secondary index covering each of those columns, and a delete costs one plus its index entries. The index part is read from INFORMATION_SCHEMA when the writer opens; a table created after that is counted without it, which the default's 16-fold headroom is there to absorb.

      Parameters:
      maxBatchCells - the cell cap, positive and at most 80,000
      Returns:
      this builder

    • maxBatchMutations

      public SpannerWriterOptions.Builder maxBatchMutations(int maxBatchMutations)
      Caps the mutations the writer puts in one batch write request. Defaults to 500.

      Every mutation costs at least one cell, so a batch never holds more mutations than cells. A value above 80,000 therefore names a batch that cannot exist and is rejected here; a value that is merely above the configured maxBatchCells(int) cannot take effect either, and build() warns about that rather than refusing it.

      Parameters:
      maxBatchMutations - the mutation cap, positive and at most 80,000
      Returns:
      this builder

    • maxBatchBytes

      public SpannerWriterOptions.Builder maxBatchBytes(long maxBatchBytes)
      Caps the estimated size of one batch write request. Defaults to 1 MiB.

      Estimated, not measured: the client library exposes no way to size a Mutation as it goes on the wire, so the writer adds up the values it can see, and it reads low. Keep the ratio to the real request limit wide enough for the estimate to be wrong in. The ceiling is a guard against a misconfiguration, not a recommendation — set there, the estimate's undercount alone puts the request over — and it is the looser of two readings of how large a batch write request may be.

      Parameters:
      maxBatchBytes - the byte cap, positive and at most 100 MiB
      Returns:
      this builder

    • maxCommitDelay

      public SpannerWriterOptions.Builder maxCommitDelay(Duration maxCommitDelay)
      Sets how long Spanner may delay a commit to group it with others, trading latency for throughput. Defaults to unset, leaving the service's own handling in place.

      Not rounded to milliseconds: the client forwards seconds and nanoseconds unchanged, so whatever is set here is what the service sees.

      Parameters:
      maxCommitDelay - the commit delay, between zero and SpannerWriterOptions.MAX_COMMIT_DELAY_LIMIT
      Returns:
      this builder

    • rpcPriority

      public SpannerWriterOptions.Builder rpcPriority(SpannerRpcPriority rpcPriority)
      Sets the priority Spanner schedules the sink's writes at. Defaults to unset, which Spanner treats as SpannerRpcPriority.HIGH.
      Parameters:
      rpcPriority - the priority
      Returns:
      this builder

    • batchWriteTimeout

      public SpannerWriterOptions.Builder batchWriteTimeout(Duration batchWriteTimeout)
      Sets the timeout for one complete batch write attempt. Defaults to 30 s.

      The timeout covers the whole server stream, including a stream that returns some group statuses and then stops making progress. The Spanner client still performs one RPC attempt; the connector's recovery* settings own every retry.

      Parameters:
      batchWriteTimeout - the per-attempt timeout, at least 1 ms
      Returns:
      this builder

    • recoveryInitialBackoff

      public SpannerWriterOptions.Builder recoveryInitialBackoff(Duration recoveryInitialBackoff)
      Sets the first backoff of the writer's retry loop. Defaults to 500 ms.
      Parameters:
      recoveryInitialBackoff - the first backoff, at least 1 ms
      Returns:
      this builder

    • recoveryMaxBackoff

      public SpannerWriterOptions.Builder recoveryMaxBackoff(Duration recoveryMaxBackoff)
      Caps the backoff of the writer's retry loop. Defaults to 10 s.
      Parameters:
      recoveryMaxBackoff - the backoff cap, at least 1 ms and at least the initial backoff
      Returns:
      this builder

    • recoveryMaxAttempts

      public SpannerWriterOptions.Builder recoveryMaxAttempts(int recoveryMaxAttempts)
      Caps the attempts of the writer's retry loop. Defaults to 10. Exhausting it fails the job — a transient failure the service never recovers from within the budget is not something a sink can drop.
      Parameters:
      recoveryMaxAttempts - the maximum attempts, positive
      Returns:
      this builder

    • build

      public SpannerWriterOptions build()
      Builds the options.

      Warns, rather than fails, when maxBatchMutations cannot take effect: a value above maxBatchCells describes a batch that cannot exist, since every mutation costs at least one cell. The configuration works — the cell cap simply decides every flush — so refusing it would reject something harmless, and saying nothing would leave a user believing they had capped a batch by count.

      Returns:
      the options