Package io.github.flink.gcp.connector.spanner.sink

Class SpannerSinkBuilder<T>

java.lang.Object
io.github.flink.gcp.connector.spanner.sink.SpannerSinkBuilder<T>
Type Parameters:
T - type of the records written by the sink
@Public public class SpannerSinkBuilder<T> extends Object
Builder for the Spanner sink; created through SpannerSink.builder().

database(DatabaseDestination) and serializer(SpannerMutationSerializationSchema) are required; everything else is defaulted.

  • Method Details

    • database

      public SpannerSinkBuilder<T> database(DatabaseDestination database)
      Sets the database to write to. Required.
      Parameters:
      database - the database
      Returns:
      this builder

    • serializer

      public SpannerSinkBuilder<T> serializer(SpannerMutationSerializationSchema<? super T> serializer)
      Sets the schema turning records into mutations. Required.
      Parameters:
      serializer - the serialization schema
      Returns:
      this builder

    • writerOptions

      public SpannerSinkBuilder<T> writerOptions(SpannerWriterOptions writerOptions)
      Sets the writer's tuning options. Optional; defaults to SpannerWriterOptions.defaults().
      Parameters:
      writerOptions - the writer options
      Returns:
      this builder

    • failedMutationHandler

      public SpannerSinkBuilder<T> failedMutationHandler(FailureHandler<? super FailedMutation> failedMutationHandler)
      Sets what happens to a mutation the service terminally refused. Optional; defaults to FailureHandler.failJob().
      Parameters:
      failedMutationHandler - the failure handler
      Returns:
      this builder

    • constraintViolationPolicy

      public SpannerSinkBuilder<T> constraintViolationPolicy(ConstraintViolationPolicy constraintViolationPolicy)
      Sets what happens to a mutation Spanner refuses for violating a constraint — a NULL in a NOT NULL column, an over-long value, a CHECK or foreign-key constraint. Optional; defaults to ConstraintViolationPolicy.FAIL_JOB.

      Under ConstraintViolationPolicy.ROUTE_TO_FAILURE_HANDLER such a mutation reaches failedMutationHandler(FailureHandler), so that handler decides whether it fails the job, is dropped or is dead-lettered.

      Parameters:
      constraintViolationPolicy - the policy
      Returns:
      this builder

    • serviceAccountKeyFile

      public SpannerSinkBuilder<T> serviceAccountKeyFile(String serviceAccountKeyFile)
      Authenticates the sink with the service-account JSON key at the given path instead of application-default credentials. The file is read on each TaskManager when its writer is created, so every TaskManager that can run the sink must see the same path. Optional; when unset the real-service path uses application-default credentials.

      Service-account keys are long-lived secrets. Prefer an attached service account or Workload Identity where the deployment supports one. This setting cannot be combined with emulatorEndpoint(String), whose plaintext channel carries no credentials.

      Parameters:
      serviceAccountKeyFile - the service-account JSON key-file path
      Returns:
      this builder

    • emulatorEndpoint

      public SpannerSinkBuilder<T> emulatorEndpoint(String emulatorEndpoint)
      Points the sink at a Spanner emulator instead of the real service. Optional; for tests.

      The emulator needs no credentials, so setting this also stops the client from looking for any.

      Parameters:
      emulatorEndpoint - the emulator's gRPC endpoint as host:port
      Returns:
      this builder
      Throws:
      IllegalArgumentException - if the endpoint is not host:port with a port in 1..65535

    • exportClientMetricsToFlink

      public SpannerSinkBuilder<T> exportClientMetricsToFlink(boolean exportClientMetricsToFlink)
      Exports the Spanner client's own metrics through Flink's metric system, so that every configured Flink metric reporter sees them. Optional; off by default.

      The writer registers them under its metric group as rpcMethod.METHOD.leaf, for the RPC methods its client calls: per method, the client's operation and attempt counts and latencies, and per attempt the Google front end's and the Spanner API front end's timing and the attempts that arrived without it. Latencies are reported as a count and a millisecond sum, so a backend divides their rates for the mean over any window; no percentile is reported. Against an emulator the client records nothing, so the series stay at zero.

      This does not stop the client's built-in export of the same metrics to Cloud Monitoring; exportClientMetricsToCloudMonitoring(boolean) does.

      Parameters:
      exportClientMetricsToFlink - whether to export the client's metrics to Flink
      Returns:
      this builder

    • exportClientMetricsToCloudMonitoring

      public SpannerSinkBuilder<T> exportClientMetricsToCloudMonitoring(boolean exportClientMetricsToCloudMonitoring)
      Sets whether the Spanner client takes part in its built-in export of client-side metrics to Cloud Monitoring. Optional; on by default, unless the SPANNER_DISABLE_BUILTIN_METRICS environment variable turns the export off.

      The client exports through one exporter per class loader, which the first client that takes part starts and which stays until the JVM exits. Off, this sink's client neither starts it nor records into it; an exporter another client started keeps running. It is independent of exportClientMetricsToFlink(boolean), which neither needs nor stops this export.

      Parameters:
      exportClientMetricsToCloudMonitoring - whether the client exports to Cloud Monitoring
      Returns:
      this builder

    • build

      public org.apache.flink.api.connector.sink2.Sink<T> build()
      Builds the sink.
      Returns:
      the sink
      Throws:
      IllegalStateException - if a required option is missing