Package io.github.flink.gcp.connector.testutils.bigquery

Class RealBigQuery

java.lang.Object
io.github.flink.gcp.connector.testutils.bigquery.RealBigQuery
@Internal public final class RealBigQuery extends Object
REST plumbing shared by the ITCases gated on real-GCP credentials: the BQ_IT_* variables, a REST client over application-default credentials, table creation/deletion and query read-back. The gated classes reach BigQuery through this and nothing else (#292); RealGcs is the sibling for the FILE_LOADS staging bucket, which only that write method needs. Here rather than in the connector's test tree so the Tier-3 BigQuery application's gated test shares it; the two helpers that take connector types stay with the connector's tests.

Deliberately a static utility and not a base class carrying the gating annotation: scripts/e2e-gated-its.sh parses test sources for the environment-variable gate and demands a surefire report per discovered file, so an abstract class carrying it would fail --assert-ran. Each gated ITCase declares its own @EnabledIfEnvironmentVariable annotations.

  • Method Summary

    Modifier and Type
    Method
    Description
    static void
    createTable(String table, com.google.cloud.bigquery.Schema schema)
    Creates table in the gated dataset with the given REST schema, which can express columns the Storage Write API schema cannot, such as INTERVAL.
    static void
    createTableWithMetadata(String table, com.google.cloud.bigquery.Schema schema, String description, Map<String,String> labels, com.google.cloud.bigquery.TableConstraints constraints)
    Creates a table carrying representative metadata that a data-only replacement must keep.
    static String
    dataset()
    The BigQuery dataset the gated ITCases create their tables in (BQ_IT_DATASET).
    static String
    datasetLocation()
    The gated dataset's location, which a reuse-enabled query source must name: BigQuery scopes a job to (project, location, id), and a location-less jobs.get sees only the US multi-region — measured 2026-08-10 against this suite's regional dataset.
    static void
    deleteTables(String... tables)
    Attempts every given table deletion and reports failures; the dataset's default table expiration (24 h, set in opentofu/flink-gcp/it-resources.tf) is the backstop for a crashed run.
    static String
    project()
    The GCP project the gated ITCases run against (BQ_IT_PROJECT).
    static long
    queryBytesProcessed(String jobId)
    Returns the bytes processed by a completed query job in the gated dataset's location.
    static List<Long>
    queryLongs(String sql)
    Runs the query and returns its first column, one entry per row, null for a NULL cell.
    static List<com.google.cloud.bigquery.FieldValueList>
    queryRows(String sql)
    Runs the query and returns its rows.
    static com.google.cloud.bigquery.TableConstraints
    tableConstraints(String table)
    Returns the constraints stored on table.
    static com.google.cloud.bigquery.StandardTableDefinition
    tableDefinition(String table)
    Returns the live definition of table — the schema, and the partitioning and clustering the service actually stored.
    static String
    tableDescription(String table)
    Returns the table-level description of table.
    static com.google.cloud.bigquery.FieldList
    tableFields(String table)
    Returns the live columns of table, in order.
    static Map<String,String>
    tableLabels(String table)
    Returns the labels stored on table.
    static String
    tablePath(String table)
    Returns the fully qualified, backtick-quoted table path for use in a query.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • project

      public static String project()
      The GCP project the gated ITCases run against (BQ_IT_PROJECT).

    • dataset

      public static String dataset()
      The BigQuery dataset the gated ITCases create their tables in (BQ_IT_DATASET).

    • datasetLocation

      public static String datasetLocation()
      The gated dataset's location, which a reuse-enabled query source must name: BigQuery scopes a job to (project, location, id), and a location-less jobs.get sees only the US multi-region — measured 2026-08-10 against this suite's regional dataset.

    • createTable

      public static void createTable(String table, com.google.cloud.bigquery.Schema schema)
      Creates table in the gated dataset with the given REST schema, which can express columns the Storage Write API schema cannot, such as INTERVAL. The connector's tests create one from a Storage Write API schema through their own RealTables.

    • createTableWithMetadata

      public static void createTableWithMetadata(String table, com.google.cloud.bigquery.Schema schema, String description, Map<String,String> labels, com.google.cloud.bigquery.TableConstraints constraints)
      Creates a table carrying representative metadata that a data-only replacement must keep.

    • deleteTables

      public static void deleteTables(String... tables)
      Attempts every given table deletion and reports failures; the dataset's default table expiration (24 h, set in opentofu/flink-gcp/it-resources.tf) is the backstop for a crashed run. A short-term table-update rate limit gets at most six attempts, with waits of one, two, four, eight and sixteen seconds. Interruption stops further cleanup.

    • tableFields

      public static com.google.cloud.bigquery.FieldList tableFields(String table)
      Returns the live columns of table, in order.

    • tableDefinition

      public static com.google.cloud.bigquery.StandardTableDefinition tableDefinition(String table)
      Returns the live definition of table — the schema, and the partitioning and clustering the service actually stored.

    • tableDescription

      public static String tableDescription(String table)
      Returns the table-level description of table.

    • tableLabels

      public static Map<String,String> tableLabels(String table)
      Returns the labels stored on table.

    • tableConstraints

      public static com.google.cloud.bigquery.TableConstraints tableConstraints(String table)
      Returns the constraints stored on table.

    • queryBytesProcessed

      public static long queryBytesProcessed(String jobId)
      Returns the bytes processed by a completed query job in the gated dataset's location.

    • queryRows

      public static List<com.google.cloud.bigquery.FieldValueList> queryRows(String sql) throws InterruptedException
      Runs the query and returns its rows.
      Throws:
      InterruptedException

    • queryLongs

      public static List<Long> queryLongs(String sql) throws InterruptedException
      Runs the query and returns its first column, one entry per row, null for a NULL cell.

      The null mapping is not decoration: FieldValue.getLongValue() is Long.parseLong(getStringValue()) and getStringValue() null-checks, so a NULL cell throws — and defaulting it to 0 would be a wrong answer rather than a failure.

      Throws:
      InterruptedException

    • tablePath

      public static String tablePath(String table)
      Returns the fully qualified, backtick-quoted table path for use in a query.