ADR-0003: Support current/previous Flink 2.x minors and Flink 1.20 LTS#

• Status: Accepted
• Date: 2026-09-10
• Supersedes: The Flink 2.3-only runtime scope in ADR-0001
• Amends: The build matrix in ADR-0002
• Unchanged-jar check amended: 2026-10-30, issue #141
• Release discovery amended: 2026-10-31, issue #142
• Current instructions: Development

Context#

The maintainer wants Flink 1.20 and 2.x support under the same policy as flink-connector-gcp. Compiling separately on each runtime establishes source compatibility but cannot prove that a released jar works on both supported 2.x minors. The 1.20 TypeInformation API still requires createSerializer(ExecutionConfig), while Flink 2.x removed it. The existing test factory therefore needs a small adapter, even though the production serializer currently compiles unchanged. Flink 1.20 also does not infer native list element information from the probe’s TypeHint.

Decision#

Supported runtimes and artifacts#

Support the current and immediately previous Flink 2.x minors with one jar compiled against the older minor, plus Flink 1.20 LTS through a separate build from the same source tree. The initial supported 2.x range is 2.2/2.3, pinned to 2.2.1/2.3.0; LTS is pinned to 1.20.4. Use JDK 17 and 21 for Flink 2.x, JDK 17 for 1.20, and Java release 17 for all output. Java 11 is unsupported.

Use Maven coordinates io.github.flink-gcp:flink-datastream-protobuf for both lines. The bare release version X.Y.Z is built at the 2.x floor, with no Flink-minor suffix. The version X.Y.Z-1.20 is compiled against Flink 1.20 with flink.compat=flink1. There is no cross-major binary compatibility guarantee. The 0.1.0, 0.2.0, and 0.3.0 capability milestones and development breaking-change policy in ADR-0001 apply to both lines. Under the 2026-09-13 publication amendment, these milestones remain unpublished; the first Maven Central versions are 1.0.0 and 1.0.0-1.20. This decision does not itself publish an artifact.

Keep one Maven module and shared sources. Use flink.compat=flink2 by default and select alternate roots only for actual API differences. Production TypeInformation uses a package-private adapter for the legacy 1.20 serializer entry point. The probe retains its own test-only adapters and is not part of the public API. Clean when switching Flink or Protobuf profiles. Validate that the selected Flink version belongs to the adapter’s major, with the 1.x adapter restricted to 1.20.

Evidence and CI#

Require full clean verification on every supported Flink/JDK combination with each paired Protobuf runtime and gencode profile. Also build each artifact line once with the default Protobuf 4 profile, the 2.x line at the floor, and run all tests with that packaged jar unchanged: at the 2.x ceiling with the same compiled tests, and with tests and runtime applications compiled against Protobuf 3 generated code and runtime, for the 2.x line at both the floor and the ceiling. Each run must load the packaged Protobuf 4 jar, verify the actual Flink and Protobuf runtimes, compare the complete class/name test inventory with the Protobuf 4 build, reject skips or failures, and verify unchanged jar and class hashes. Run these binary checks on JDK 17. A jar built with Protobuf 3 is not a release artifact, so it needs no binary check. The source-build matrix covers JDK 21; it does not substitute for the binary check.

This single-module library runs the supported matrix and binary checks on every PR as well as main pushes, manual runs, and a weekly schedule. This is more frequent than the connector’s weekly compatibility lanes while preserving its support contract. The scheduled run checks the pinned supported versions; it is not automatic discovery of new Flink releases or a next-SNAPSHOT early-warning lane. A separate weekly upstream release check outside the PR gate reports upstream releases that overtake the pins. It does not move a pin or the window; each update remains the reviewed change described here. Before moving the supported window, explicitly review the new release, advance the floor and ceiling together, update documentation, and pass the matrix and binary checks. Dependabot must not independently advance Flink minor or major versions; patch updates remain reviewable. Support for a newly released minor starts with that reviewed range update, not merely its upstream publication.

The tests cover the production serializer, TypeInformation and factory selection, and native transport, alongside a separate test-only factory/transport serializer. The runtime acceptance suite also verifies checkpoint recovery and separate-job canonical savepoint restoration with HashMap and RocksDB backends, including restoration across supported schema changes. Development fixtures retain a baseline for each pinned Flink/Protobuf/backend combination. All supported runtimes must retain native top-level, POJO, tuple, Row, and explicit List transport with generic types disabled. The inferred List control must retain its version-specific result: native element information on 2.x, generic fallback and serializer rejection on 1.20. Users on 1.20 must supply explicit list element TypeInformation.

State and release acceptance#

Issues #9, #10, and #11 must apply their production integration and unchanged-schema restore acceptance to each supported runtime and artifact line. The release work in #13 must publish and verify both version lines, retain per-line published-artifact provenance, and keep released-state fixtures for each supported runtime. Before publication, guard the effective Maven model: bare versions must use the exact recorded 2.x floor and flink2; -1.20 versions must use the pinned 1.20 toolchain and flink1. Checking only command-line substrings is insufficient because Maven properties can be overridden. Both staged artifacts must validate before either is published; publish both before announcing the GitHub release. The two publications are not transactional, so the release workflow must record partial failures and recovery instructions; ADR-0006 records how it does.

Source and binary compatibility do not imply saved-state compatibility across Flink versions. Library-only upgrade fixtures keep the Flink version fixed and use the matching artifact line. Cross-Flink-minor or cross-major savepoint upgrades require a separately declared path and fixtures proving restored values and continued processing; no such path is claimed here. Versioned descriptor snapshots replace the intermediate snapshotConfiguration() failure. All claimed runtime restore paths must still be verified before 1.0.0 publication.

Source basis and alternatives#

The policy follows flink-connector-gcp at commit 40967023bcd27adbd81b7c659207d6fe791dbdef:

• ADR-0053: One artifact covers current and previous Flink minors .
• ADR-0054: Flink 1.20 from the same source tree .
• ADR-0147: Two version lines and publication validation .

Separate branches or modules would duplicate the shared serializer before there is enough incompatible code to justify them. A single jar promised across Flink majors would claim more than the source-build checks establish. Compiling only against the newest 2.x minor would permit accidental use of APIs absent from the older supported runtime.