Package io.github.flink.gcp.connector.bigquery.sink

Enum OrderingMode

java.lang.Object
java.lang.Enum<OrderingMode>
io.github.flink.gcp.connector.bigquery.sink.OrderingMode
All Implemented Interfaces:
Serializable, Comparable<OrderingMode>, java.lang.constant.Constable
@Experimental public enum OrderingMode extends Enum<OrderingMode>
Whether the WriteMethod.STORAGE_API_AT_LEAST_ONCE writer commits a table's append requests in the order a sink subtask sent them, so that no row commits ahead of a row the subtask received earlier and sent in an earlier request.

The order is observable to a reader that depends on the commit order: a query over APPENDS that keeps the row with the latest _CHANGE_TIMESTAMP per key, a continuous query that forwards rows as they commit, or a reader that resumes after the highest value it has seen and never returns for a row that lands behind it. It does not decide the value a CDC table settles on: every CDC row carries a _CHANGE_SEQUENCE_NUMBER, and BigQuery applies one primary key's mutations by it (ADR-0193), except where two of a key's mutations carry equal sequences, which BigQuery resolves by ingestion time.

BigQuery's CDC documentation states no order among the rows of one append request, and, measured without a sequence, the first of a key's changes in one request won. Equal sequences come from a source that can repeat one, or from Flink's ChangelogNormalize, which completes a key-only delete from the last row it holds and so gives it the sequence of the change it deletes. For a CDC write, this mode therefore also keeps one primary key out of a request twice: the writer sends the pending request before a row whose key it already holds, and a key changed in quick succession costs a request per change. The key is read from the bytes the serializer wrote, and a key column written in an encoding with several spellings of one value, such as a TIMESTAMP or NUMERIC written as text, makes every row of the table its own request.

Each subtask orders only what it receives, so the order of rows that reach different subtasks stays undefined whatever this mode says. A key's rows in different requests keep their order only when every row of the key reaches the same subtask: a Table API changelog that Flink shuffles by the sink's primary key, or a stream keyed before the sink.

See Also:

  • Enum Constant Details

    • NONE

      public static final OrderingMode NONE
      No order between the append requests of one table (default). The writer keeps several requests in flight, and re-sends a failed request on a rebuilt stream writer after later requests may have committed, so a table's rows can commit out of the order they arrived.

    • PER_TABLE

      public static final OrderingMode PER_TABLE
      Commits a table's append requests in the order the subtask sent them. The writer sends a table's next append request only after the previous one has committed, or after its failed rows have been re-sent or handed to the failure handler; different tables still have their requests in flight together. A table therefore has one request in flight at a time, so its throughput is bounded by one round trip per request of up to maxAppendRequestBytes, and an attempt that timed out on the client can still be committed by the service after a later request. For a CDC write, a request also ends before a row whose primary key it already holds.

  • Method Details

    • values

      public static OrderingMode[] values()
      Returns an array containing the constants of this enum type, in the order they are declared.
      Returns:
      an array containing the constants of this enum type, in the order they are declared

    • valueOf

      public static OrderingMode valueOf(String name)
      Returns the enum constant of this type with the specified name. The string must match exactly an identifier used to declare an enum constant in this type. (Extraneous whitespace characters are not permitted.)
      Parameters:
      name - the name of the enum constant to be returned.
      Returns:
      the enum constant with the specified name
      Throws:
      IllegalArgumentException - if this enum type has no constant with the specified name
      NullPointerException - if the argument is null

    • toString

      public String toString()
      Returns the spelling a WITH clause uses.
      Overrides:
      toString in class Enum<OrderingMode>