Gated E2E audit (September 2026)#
This records the source audit for #1346, within the weekly E2E repair. The inventory contains 46 concrete gated classes: 42 selected by the weekly recipe and four separately approved manual classes. Every row received a static review of gates, provisioning/ownership, waits and cancellation, assertion strength, and cleanup; inherited helpers are part of that review. A static review is not evidence that a real-service run passed. Runtime results belong to the associated run and PR records; the manual classes below are excluded from this repair’s billed execution.
Shared repairs#
Pub/Sub, Bigtable and Spanner teardown now reports resource-deletion failures and still closes clients.
Only the service’s not-found result is accepted as an already-absent resource.
Pub/Sub records owned resource names before creation, so an ambiguous create failure still has a cleanup target.
Bigtable attempts every table’s retention update before reporting collected failures; deletion still requires retention to have been disabled.
The native staged-sink probe waits for every execution vertex to be running before requesting its stop-savepoint; a complete admission count alone does not establish that readiness.
BigQuery table cleanup attempts every named table unless interrupted, and a failed table cleanup does not skip a separate staging-prefix cleanup.
The observed HTTP 403/rateLimitExceeded deletion response receives at most six attempts with exponential backoff; exhaustion and other errors remain visible, and interruption stops further deletions.
Cloud Storage deletion continues after an individual object’s deletion fails; a failed listing can still prevent discovering further objects.
The Pub/Sub polling helper bounds each pending pull to 30 seconds and cancels that future on timeout or interruption. The observation deadline is checked between completed pulls, so its last pull may extend the observation by up to 30 seconds. A timed-out pull fails the test; it cannot establish that a dead-letter subscription is empty. Acknowledgement and other administrative RPCs retain their SDK bounds; this is not a deadline around every RPC in a test. Its client cleanup attempts every client/channel release and preserves failures. Credential-free regressions exercise pending/completed/failed/interrupted pulls, client-close aggregation, table-delete aggregation and retention-update aggregation. Direct Spanner transaction scopes explicitly invoke cleanup and close. In the audited Spanner SDK (6.121.0), those transactions use multiplexed sessions and cleanup does not delete their service session. Spanner job cleanup observes the local job result before cancelling: completed local jobs can already have shut down their MiniCluster. A cancellation failure is accepted only if job termination is observed within the bounded wait; otherwise the failure propagates.
Gated methods that used the default same-thread timeout now use SEPARATE_THREAD, including the inherited Cloud Tasks recovery methods through a concrete-class override.
This follows ADR-0119’s existing interruption-resistant iterator finding.
The method timeout does not cover provisioning lifecycle methods or guarantee termination of an abandoned test thread.
The integration-test fork ceiling and external resource sweep remain necessary, and a hard kill can bypass Java cleanup.
No test reruns, skips, weaker assertions or production connector APIs are introduced.
Class inventory#
“Weekly” means selected by the five ordinary service gates; “Manual” means static audit only in this repair. Shared cleanup changes apply even where a class needed no direct edit.
| Class | Execution scope | Reviewed contract and disposition |
|---|---|---|
BigQueryFileLoadsITCase | Weekly | Load-job/readback and staging ownership; table and object cleanup both attempted. |
BigQueryFileLoadsSchemaEvolutionITCase | Weekly | Load-job/readback and staging ownership; table and object cleanup both attempted. |
BigQueryFileLoadsStreamingITCase | Weekly | Load-job/readback and staging ownership; table and object cleanup both attempted. |
BigQueryLoadJobRunnerRealGcpITCase | Weekly | Load-job/readback and staging ownership; table and object cleanup both attempted. |
BigQueryBufferedStreamExactlyOnceITCase | Weekly | Checkpoint, restart and savepoint readback; cleanup failures now propagate. |
BigQueryDefaultStreamAtLeastOnceITCase | Weekly | Fan-out and restart distinctness; both cleanup groups attempted. |
BigQueryBufferedStreamMissingTableITCase | Weekly | Existing-table permission control and absent-table refusals; all cleanup groups attempted. |
BigQueryBufferedStreamSchemaEvolutionITCase | Weekly | Schema/offset continuity and readback; writer and committer independently released. |
BigQueryBufferedStreamSchemaPropagationITCase | Manual | Propagation measurement retains schema and metric oracles; writer and committer independently released. |
BigQueryDefaultStreamSchemaEvolutionITCase | Manual | Propagation diagnostics retained; logger restored, probe threads checked and writer released. |
BigQuerySerializerFidelityITCase | Weekly | Schema, row or filter/readback assertions retained; separate-thread timeout and shared table cleanup. |
BigQueryMultiStreamRealGcpITCase | Weekly | Multiple-reader participation, snapshot-pinned count and deliberate restart; a checkpoint gate keeps failure injection reachable on fast reads. |
BigQueryQuerySourceRealGcpITCase | Weekly | View/query and result ownership paths; view failure cannot prevent table cleanup. |
BigQuerySourceRealGcpITCase | Weekly | Schema, row or filter/readback assertions retained; separate-thread timeout and shared table cleanup. |
BigQueryCdcAutoCreateRealGcpITCase | Weekly | Schema, row or filter/readback assertions retained; separate-thread timeout and shared table cleanup. |
BigQueryTableCreationFidelityITCase | Weekly | Schema, row or filter/readback assertions retained; separate-thread timeout and shared table cleanup. |
BigQueryTableExactlyOnceITCase | Weekly | Row count and distinctness through Table API; periodic checkpoints configured, no forced recovery claim. |
BigQueryTableFileLoadsITCase | Weekly | Load-job/readback and staging ownership; table and object cleanup both attempted. |
BigQueryTableSourceFidelityITCase | Weekly | Schema, row or filter/readback assertions retained; separate-thread timeout and shared table cleanup. |
BigQueryFilterPushDownRealGcpITCase | Weekly | Schema, row or filter/readback assertions retained; separate-thread timeout and shared table cleanup. |
BigtableAsyncSqlRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableStagedSinkRealGcpITCase | Weekly | Service/readback and replay oracles retained; stop-savepoint waits for all execution vertices to run, and shared cleanup hardened. |
BigtableConditionalRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableAutoCreationRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableRejectionRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableSinkRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableReadModifyWriteRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableChangeStreamSourceRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableSourceRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableAggregateTableRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableTableSinkRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
BigtableTableSourceRealGcpITCase | Weekly | Service/readback oracles and existing separate-thread timeout retained; shared retention, instance and client cleanup hardened. |
CloudTasksRecoveryRealGcpITCase | Manual | Inherited staged recovery, rescaling, no-send-before-checkpoint and expiry oracles; separate-thread timeout override. |
CloudTasksTombstoneRealGcpITCase | Manual | Explicit retention acceptance and cleanup evidence; separate-thread method timeout. |
CloudTasksAppEngineRealGcpITCase | Weekly | Task readback and HTTP/App Engine semantics; separate-thread method timeout. |
PubSubPublisherLifecycleRealGcpITCase | Weekly | Payload, ordering or service-refusal assertions retained; shared subscription/topic/client cleanup hardened. |
PubSubSinkRejectionRealGcpITCase | Weekly | Payload, ordering or service-refusal assertions retained; shared subscription/topic/client cleanup hardened. |
PubSubDeadLetterRealGcpITCase | Weekly | Exact poison payload forwarded; only NOT_FOUND means absent; cancellation failure remains visible. |
PubSubSourceOrderingRealGcpITCase | Weekly | Payload, ordering or service-refusal assertions retained; shared subscription/topic/client cleanup hardened. |
PubSubSubscriptionAdminIamRealGcpITCase | Weekly | Permission-denied and existence controls; clients closed and unexpected creations registered. |
PubSubSubscriptionAdminRealGcpITCase | Weekly | Subscription policy/ownership assertions; registration precedes creation attempts. |
PubSubBackpressuredSplitBufferRealGcpITCase | Weekly | Callback/overflow/ordered-redelivery/empty-DLQ oracles; readiness repair is tracked separately in #1345. |
PubSubNackPromptnessRealGcpITCase | Weekly | Prompt redelivery through the streaming-pull split reader; shared fixture cleanup hardened. |
SpannerRejectionRealGcpITCase | Weekly | Mutation and size refusals; an empty callback stream cannot establish acceptance. |
SpannerSourceRealGcpITCase | Weekly | Partition coverage and change-stream recovery; explicit transaction cleanup/close scopes and visible job cancellation failures. |
SpannerFloatKeyReadRealGcpITCase | Weekly | Candidate-key reads compared with reference rows, including signed-zero and residual-filter cases. |
Reviewed helper boundaries#
The review includes RealBigQuery, RealGcs, PubSubTestClients, the three abstract real-GCP fixtures, Awaits, Drains, Bigtable’s staged-job lifecycle and replay observations, Spanner’s change-stream observers and float-key read support, and Cloud Tasks’ inherited StagedRecoveryAcceptance and acceptance cleanup.
Existing observation-specific limits remain visible: a count/sum probe does not establish byte-for-byte identity, a configured periodic checkpoint is not a forced-recovery experiment, and a diagnostic schema-propagation canary is not the connector’s acceptance oracle.
Bigtable’s replay oracle continues to require the original identity-to-row inventory and ALREADY_COMMITTED observations; unchanged readback alone does not satisfy it.
Cloud Tasks’ separate manual gate, paused-queue admission and retention readback remain intact.
The shared polling helpers rely on the condition/iterator owner to cancel work.
Awaits cannot preempt a blocking condition; Drains delegates iterator closure to its caller.
These local limits do not replace the class, fork or external sweep boundaries.
The acceptance criteria for #1344 require two consecutive successful full GitHub E2E runs after merge, followed by success of the next actual scheduled E2E and its post-run sweep.