Testing#

The suites fall into three kinds, split by what they need to run: nothing beyond Docker, real Google Cloud credentials, or wall clock. just verify runs the first kind and excludes the other two, so an ordinary build is credential-free and its duration is bounded.

Unit and integration tests#

Name a unit test *Test and an integration test *ITCase. The build selects by the first suffix alone: a class ending in Test runs in the unit execution, and every other test class lands in the integration execution — an unconventional name does not escape the build, it runs in the wrong lane. Both kinds run in just verify. The integration tests talk to Docker-backed service emulators, so Docker must be running, but no Google Cloud project is touched. An emulator is a convenience rather than an authority: the deviations that have been measured are recorded on the connector pages, and the real service is exercised by the gated suites below.

While iterating, scope the run to what changed rather than rebuilding the reactor — and keep -am, so the sibling modules come from the working tree rather than from whatever jars an earlier build installed. For a single-module change, the per-connector CI lane carries the full verification:

./mvnw test -pl flink-connector-gcp-pubsub -am -Dtest=PubSubSinkBuilderTest -Dsurefire.failIfNoSpecifiedTests=false
just verify-module flink-connector-gcp-pubsub    # the module-wide build, when a broader check is wanted

Emulator image updates#

The Renovate emulator images workflow proposes a grouped draft pull request on the first day of each month at 03:23 UTC. It updates the image constants in test-utils; Bigtable, Pub/Sub and Firestore share a pin and move together. Review the Bigtable and Firestore deviation suites’ verdicts and, for a Spanner bump, recheck the measurements named in the pull request before merging. A green emulator suite does not establish real-service behavior.

For an initial run or recovery, dispatch the workflow on main:

gh workflow run renovate.yaml --repo flink-gcp/flink-connector-gcp --ref main
gh run list --repo flink-gcp/flink-connector-gcp --workflow renovate.yaml

Check the run’s logs for lookup failures and inspect any resulting update pull request and its CI runs. Missing App credentials, reported Docker package lookup failures, missing or invalid repository configuration, and the Host error / Git error - aborting paths fail the job. An up-to-date set of images can finish successfully without opening a pull request; an existing update pull request can be refreshed instead. The monthly run does not replace reviewing and merging updates, and no separate retention checker watches a pin left unmerged.

Credential-gated suites#

A test that talks to real Google Cloud carries @Tag("gated") together with an environment-variable gate naming the project it bills. Ordinary builds exclude the tag, so just verify never runs these — even in a shell that holds the variables. The opt-in is:

just e2e

It needs an authenticated gcloud CLI besides the environment gates — the one tool this page names that mise does not install; the App Engine fixture and just sweep-e2e both call it. It refuses to start unless the environment gates are set, runs the gated classes its six per-service gates select, and asserts afterwards that they actually ran, so a silently skipped suite cannot pass as a green one. A few deliberately manual cases — slow schema-propagation observations — sit behind gates of their own, outside just e2e.

The gated classes of the Tier-3 applications under kubernetes/apps join the same run. Each application sits outside the default reactor, so the recipe installs its connector from the working tree and then builds it behind its tier3-<name> profile; the BigQuery application’s FILE_LOADS savepoint restore is one such class.

One gate runs alone with its gate variable as the argument, which requires only that gate’s variables and builds only its modules:

just e2e-gate PUBSUB_IT_PROJECT

scripts/e2e-gated-its.sh --gates lists the gates.

just e2e runs the gates one after another, and each gate runs even when an earlier one fails. It does not run them in parallel, because each module’s integration tests keep two JVMs with a 2 GiB heap each, and six modules at once would need about 24 GiB on a workstation. The E2E workflow runs one job per gate in parallel instead, so a run takes about as long as the Bigtable suite, the longest. No gate waits for another: the Cloud Tasks gate starts and stops its App Engine fixture around its own class and then verifies the stopped, zero-instance state, and a failed verification fails that gate rather than holding back the rest.

Each gate removes its selected old reports before starting and validates its full XML reports against its own inventory at exit, including on failure. Missing, stale, truncated, mismatched, failed or skipped results fail the gate. The run then reads every gate’s evidence back against the whole inventory, so a gate whose job was cancelled, skipped or never started fails the run instead of being left out of it. In CI it also requires every gate’s job to have succeeded, because evidence alone cannot show a gate that failed after its tests passed, such as a failed App Engine stop. target/e2e/evidence/<gate>/ contains each gate’s per-class results and summary, and target/e2e/evidence/ the combined results and summary; GitHub Actions retains both for 14 days. The exported evidence contains selected method names, outcomes and Java stack frames; it omits report properties, environment values and arbitrary test output. Full test output remains in the job log.

These suites create and delete billed resources; each run costs real money, which is why CI runs them on a weekly schedule (plus manual dispatch) rather than per pull request.

The Pub/Sub hard-buffer-limit probe waits up to 120 seconds for the two callbacks needed to cross its one-message capacity, then gives the limit event and subscriber stop a separate 15-second deadline. A timeout reports which phase failed, the SDK state and the observed callback and buffer counts. The probe still requires the original ordered sequence to be redelivered and the dead-letter observer to remain empty.

The Google Cloud project behind them is provisioned in two layers. The persistent layer — service accounts, Workload Identity Federation, buckets, the dataset and the Tier-3 cluster — is OpenTofu under opentofu/, whose README documents the layout, the CI plan/apply flow and the security model (no service-account keys; local runs use your own application-default credentials). The runs themselves create what they test against — tables, topics, subscriptions and queues, but also one ephemeral Bigtable or Spanner instance per gated class and a briefly started App Engine version for Cloud Tasks, which are the resources that cost money while they stand. A completed run deletes them; just sweep-e2e returns what an interrupted run left standing — stale instances, the serving App Engine version — to its idle state. Its default spares instances younger than two hours (on a schedule, age is what tells a leak from a run still using its instance), so cleaning up right after an interrupted local run means just sweep-e2e --all — which drops that safeguard entirely, so say it only when nothing else, scheduled or local, is using the project. The environment gates come from an uncommitted .env at the repository root; just worktree-env makes it reachable from a git worktree.

The on-demand GKE Autopilot rig (#38) has a standing cluster management fee, even between test sessions. Availability of the billing account’s shared free-tier credit is unverified. Its runbook records costs, separate resource approval and cleanup. just e2e and just sweep-e2e do not launch or clean up this rig. Its Operator installation and CUE-managed workloads follow the cloud foundation; routine suites continue to use MiniCluster. The lifecycle runbook defines its scenarios: a smoke run and the BigQuery, Pub/Sub and Cloud Tasks recovery trials, each dispatched on demand. They are correctness exercises, not a release gate or a performance benchmark. The Cloud Tasks trial checks the staged sink’s recovery; the Cloud Tasks performance assessment for #1246 ran on a single VM rather than on this rig.

The slow lane#

A test whose duration is the instrument — an elapsed-time observation that cannot be tuned down without destroying what it measures — carries @Tag("slow"). It is excluded from just verify alongside the gated tag and runs in the weekly workflow instead, so the per-change build stays fast without the coverage silently disappearing. Running it locally means narrowing the exclusion back to the gated tag alone:

just verify -Dtest.excluded.groups=gated

The September 2026 gated E2E audit records the 46-class static review, cleanup and deadline repairs, and the boundary between the weekly suite and manual acceptance probes.