Package io.github.flink.gcp.connector.pubsub.sink

Class PubSubPublisherOptions

java.lang.Object
io.github.flink.gcp.connector.pubsub.sink.PubSubPublisherOptions
All Implemented Interfaces:
Serializable
@Public public final class PubSubPublisherOptions extends Object implements Serializable
Tuning options for the sink's Pub/Sub publishers and its writer: SDK batching and publish-retry settings, message ordering, the writer's in-flight caps, the backoff budget of the topic auto-creation recovery, and the writer's shutdown budget.

Set via PubSubSinkBuilder.publisherOptions(PubSubPublisherOptions); optional — every knob left unset keeps the SDK's (or the sink's) default behavior, so defaults() is equivalent to not setting options at all.

In-flight publishes are bounded by the writer itself, along both dimensions that matter: PubSubPublisherOptions.Builder.maxInFlightMessages(int) and PubSubPublisherOptions.Builder.maxInFlightBytes(long). Both yield to the task mailbox rather than blocking the task thread, and both apply with message ordering enabled. The SDK publisher's own flow controller is deliberately not exposed; the connector documentation records why.

Instances are immutable and serializable.

See Also:

  • Field Details

    • DEFAULT_MAX_CONSECUTIVE_REJECTIONS

      public static final int DEFAULT_MAX_CONSECUTIVE_REJECTIONS
      The default PubSubPublisherOptions.Builder.maxConsecutiveRejections(int): enough confirmed rejections in a row to say the stream's data is broken rather than anomalous, at an isolation cost of about a hundred solo publishes — one Publish round trip each — before the job fails.
      See Also:

    • DEFAULT_MAX_ACTIVE_PUBLISHERS

      public static final int DEFAULT_MAX_ACTIVE_PUBLISHERS
      The default maximum number of publishers retained by one sink writer subtask.
      See Also:

    • DEFAULT_DESTINATION_IDLE_TIMEOUT

      public static final Duration DEFAULT_DESTINATION_IDLE_TIMEOUT
      The default time an unused publisher remains active before a checkpoint releases it.

    • UNBOUNDED

      public static final int UNBOUNDED
      PubSubPublisherOptions.Builder.maxConsecutiveRejections(int) value under which the bound never fires.
      See Also:

  • Method Details

    • builder

      public static PubSubPublisherOptions.Builder builder()
      Creates a new PubSubPublisherOptions.Builder.
      Returns:
      a new builder

    • defaults

      public static PubSubPublisherOptions defaults()
      Returns the default options: every knob left where PubSubPublisherOptions.Builder initialises it, which is the SDK's own default wherever the sink has no reason to pick one.

      The values themselves are on the reference/pubsub.md table, one row per setter, rather than enumerated here.

      Returns:
      the default options

    • getBatchElementCountThreshold

      @Nullable public Long getBatchElementCountThreshold()
      Returns the batch element-count threshold, or null for the SDK default.

    • getBatchRequestByteThreshold

      @Nullable public Long getBatchRequestByteThreshold()
      Returns the batch request-byte threshold, or null for the SDK default.

    • getBatchDelayThreshold

      @Nullable public Duration getBatchDelayThreshold()
      Returns the batch delay threshold, or null for the SDK default.

    • getRetryTotalTimeout

      @Nullable public Duration getRetryTotalTimeout()
      Returns the publish-retry total timeout, or null for the SDK default.

    • getRetryInitialDelay

      @Nullable public Duration getRetryInitialDelay()
      Returns the initial publish-retry delay, or null for the SDK default.

    • getRetryDelayMultiplier

      @Nullable public Double getRetryDelayMultiplier()
      Returns the publish-retry delay multiplier, or null for the SDK default.

    • getRetryMaxDelay

      @Nullable public Duration getRetryMaxDelay()
      Returns the maximum publish-retry delay, or null for the SDK default.

    • getRetryInitialRpcTimeout

      @Nullable public Duration getRetryInitialRpcTimeout()
      Returns the initial per-RPC timeout, or null for the SDK default.

    • getRetryRpcTimeoutMultiplier

      @Nullable public Double getRetryRpcTimeoutMultiplier()
      Returns the per-RPC timeout multiplier, or null for the SDK default.

    • getRetryMaxRpcTimeout

      @Nullable public Duration getRetryMaxRpcTimeout()
      Returns the maximum per-RPC timeout, or null for the SDK default.

    • getRetryMaxAttempts

      @Nullable public Integer getRetryMaxAttempts()
      Returns the maximum publish attempts, or null for the SDK default (bounded only by the total timeout).

    • isEnableMessageOrdering

      public boolean isEnableMessageOrdering()
      Returns whether publishers honor message ordering keys.

    • getMaxInFlightMessages

      public int getMaxInFlightMessages()
      Returns the writer's cap on unacknowledged publishes.

    • getMaxInFlightBytes

      public long getMaxInFlightBytes()
      Returns the writer's cap on the serialized bytes of unacknowledged publishes.

    • getRecoveryInitialBackoff

      public Duration getRecoveryInitialBackoff()
      Returns the first backoff of the topic auto-creation recovery.

    • getRecoveryMaxBackoff

      public Duration getRecoveryMaxBackoff()
      Returns the backoff cap of the topic auto-creation recovery.

    • getRecoveryMaxAttempts

      public int getRecoveryMaxAttempts()
      Returns the maximum republish attempts of the topic auto-creation recovery.

    • getPublishProgressTimeout

      public Duration getPublishProgressTimeout()
      Returns how long the writer waits with no publish completing before it fails.

    • getShutdownTimeout

      public Duration getShutdownTimeout()
      Returns how long one publisher release waits for shutdown.

    • getMaxActivePublishers

      public int getMaxActivePublishers()
      Returns the maximum number of publishers retained by one sink writer subtask.

    • getDestinationIdleTimeout

      public Duration getDestinationIdleTimeout()
      Returns how long an unused publisher remains active before a checkpoint releases it.

    • isPerDestinationMetrics

      public boolean isPerDestinationMetrics()
      Returns whether the writer registers per-topic send counters.

    • getMaxConsecutiveRejections

      public int getMaxConsecutiveRejections()
      Returns how many consecutive confirmed rejections fail the job, or UNBOUNDED for none.

    • toRecoverySchedule

      @Internal public RetrySchedule toRecoverySchedule()
      Returns the topic auto-creation recovery schedule the recovery* knobs describe. Jittered: every subtask that parked publishes for the same missing topic resumes against the same freshly created topic, so unjittered they would republish in lockstep.

    • hasBatchingOverrides

      public boolean hasBatchingOverrides()
      Returns whether any batching knob deviates from the SDK default.

    • hasRetryOverrides

      public boolean hasRetryOverrides()
      Returns whether any publish-retry knob deviates from the SDK default.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object