Package io.github.flink.gcp.connector.bigquery.source

Class BigQuerySourceConfig<T>

java.lang.Object
io.github.flink.gcp.connector.bigquery.source.BigQuerySourceConfig<T>
Type Parameters:
T - type of the records produced by the source
All Implemented Interfaces:
Serializable
@Internal public final class BigQuerySourceConfig<T> extends Object implements Serializable
Everything the source's enumerator and readers were configured with, as one immutable object shipped inside the job graph.
See Also:

  • Method Details

    • getTable

      @Nullable public TableDestination getTable()
      Returns the table being read, or null when a query decides it.

      Exactly one of this and getQuery() is set; which one is what the builder enforced.

    • getQuery

      @Nullable public String getQuery()
      Returns the query whose result is read, or null when a table is named directly.

    • getQueryLocation

      @Nullable public String getQueryLocation()
      Returns the location the query job runs in, or null to let BigQuery infer it.

    • getQueryResultDataset

      @Nullable public String getQueryResultDataset()
      Returns the dataset the query's result is written to, or null for BigQuery's own anonymous dataset.

    • getReuseQueryResultWithin

      @Nullable public Duration getReuseQueryResultWithin()
      Returns how long a re-planned job may reuse a previous attempt's query job, or null for the default: a random job id, under which nothing is ever reused.

    • isMaterializeViewsEnabled

      public boolean isMaterializeViewsEnabled()
      Returns whether materializeViews() was asked for.

      Named ...Enabled rather than after the field alone, which every other boolean getter here is: isMaterializeViews() reads as a question about a materialized view — a real BigQuery noun, and one this very class decides about — rather than as "the option is on". A reader misread it that way, which is the whole argument.

      Off unless asked for, because deciding it costs a metadata call the read path otherwise never makes.

    • getQueryRunner

      @Nullable public QueryRunner getQueryRunner()
      Returns the seam running the query, or null when a table is named directly.

    • describeInput

      public String describeInput()
      Returns what this source reads, as it reads mid-sentence, for the enumerator's log lines and failure messages.

      The query is not quoted in full: it can be arbitrarily long, and a message that buries its own point under a page of SQL is worse than one that names the job to look up.

    • getParentProject

      public String getParentProject()
      Returns the project the read session belongs to and is billed to.

    • getDeserializer

      public BigQueryRowDeserializationSchema<T> getDeserializer()
      Returns the deserializer converting rows into records.

    • getSelectedFields

      public List<String> getSelectedFields()
      Returns the columns read, or an empty list for every column.

    • getRowRestriction

      @Nullable public String getRowRestriction()
      Returns the server-side row filter, or null for no filter.

    • getSnapshotTime

      @Nullable public Instant getSnapshotTime()
      Returns the instant the table is read as of, or null for its current contents.

    • getMaxStreamCount

      public int getMaxStreamCount()
      Returns the upper bound on read streams, or 0 to let BigQuery decide.

    • getPreferredMinStreamCount

      public int getPreferredMinStreamCount()
      Returns the preferred lower bound on read streams, or 0 for none.

    • getMaxRecordsPerFetch

      public int getMaxRecordsPerFetch()
      Returns the most rows one fetch hands to the task thread.

    • getMaxBytesPerFetch

      public long getMaxBytesPerFetch()
      Returns the target serialized Avro bytes one fetch hands to the task thread.

    • getSessionCreatorFactory

      public ReadSessionCreatorFactory getSessionCreatorFactory()
      Returns the factory the source mints one session creator per enumerator from.

      A factory rather than a creator because the JobManager holds one source object for a job's whole life, so a creator here would be shared by every enumerator a coordinator reset builds and the first teardown would refuse every later one (docs/adr/0128).

    • getRowStreamOpener

      public RowStreamOpener getRowStreamOpener()
      Returns the seam opening read streams.