Aliased large-frame reads in the native serializer#

This report measures the large-frame read path changed for #113 . After the generated-size change , large deserialize and deserializeReuse were the only cells that missed the ADR-0004 native/Chill tolerance on both profiles, at 1.26 to 1.27 times Chill’s time with about 132 KB/op allocated for a 64 KiB payload. One plan on temporary GCE VMs compares the change with the unchanged serializer. The measurement contract is ADR-0004 ; the comparison does not establish whole-job throughput, heap retention, or state compatibility.

Change under test#

Before the change, a payload above 4 KiB was parsed by Protobuf’s stream decoder, which in both pinned runtimes copies a bytes value larger than its 4 KiB buffer twice. The changed serializer copies such a payload through its retained 4 KiB array into a new array, which starts at no more than 1 MiB and doubles until it holds the payload, and parses it with aliasing enabled, so decoded bytes fields, proto2 strings, and unknown length-delimited fields may share that array; proto3 strings are still decoded into new String values. Payloads of at most 4 KiB keep the retained scratch array and the non-aliasing array decoder, and writes are unchanged. ADR-0001 records the amended read-side rule, the decoder it requires, its retention trade-off, and the alternatives.

The benchmark record has one bytes field, a string, a nested message, a packed repeated int64, and a map. Only the large workload, whose bytes field holds 65,536 random bytes, has a payload above 4 KiB; the text workload’s payload is below it, so the other workloads read through the unchanged scratch-array strategy, whose read loop and end-of-frame check this change simplifies.

Method#

The plan used the temporary GCE procedure in project flink-gcp. Each Protobuf profile ran on its own c3-standard-8 VM in asia-northeast1-b (Intel Xeon Platinum 8481C, Sapphire Rapids, 8 vCPUs, 32 GiB) from image ubuntu-2404-noble-amd64-v20261024, kernel 7.0.0-1014-gcp, with Temurin 17.0.21, Flink 2.3.0, and Protobuf/protoc 3.25.9 or 4.33.6. Both VMs recorded TERMINATE host maintenance and NONE as the only maintenance event, passed the load gate before every build and measurement, and reported the same machine type, CPU platform, image, kernel, and JDK. just benchmark-pair ran the ADR-0004 long-run protocol for the native and Chill Protobuf lanes, and each variant took about 70 minutes.

The baseline is the previous serializer and its tests, identical to the base under src; the candidate is the source commit unchanged.

Comparisons between the variants use the Chill lane as a per-block control, as in the earlier reports: the ratio of the two variants’ geometric-mean native/Chill time ratios, with a pooled two-sample Student-t interval on the five log ratios of each run (8 degrees of freedom). Below 1 favors the candidate. Allocation is the five-fork mean of JMH’s normalized allocation, in bytes per operation. These intervals are per cell, without a simultaneous claim across cells.

Results#

Reading and copying#

Each variant cell is native time in ns/op and allocation in B/op, as five-fork means. The last three columns give the candidate’s time relative to the baseline and each variant’s native/Chill interval and ADR-0004 verdict.

Protobuf 4:

Protobuf 3:

On both profiles, large deserialize and deserializeReuse take 0.433 to 0.436 times the baseline’s time, with every interval between 0.426 and 0.443. They allocate 65,969 to 65,988 B/op, against 132,056 to 132,080 B/op before; what remains is about the 64 KiB frame array plus the decoded message. Their native/Chill ratios move from 1.253 to 1.261, missed, to 0.544 to 0.549, established. Every other reading and copying cell stays established, and the small-frame reads allocate the same in both variants except Protobuf 3 collections deserializeReuse, at 3,816 and 3,611 B/op, within the 3,587 to 3,816 B/op that the generated-size report recorded for the same scratch-array reads.

Writes#

Writes run identical code in both builds.

Protobuf 4:

Protobuf 3:

Every write cell of both variants is established, except Protobuf 3 text serialize in the baseline run, which is inconclusive at 0.580 (0.247, 1.362). The standard deviation of the five log native/Chill ratios is 0.38 to 0.69 for the three Protobuf 3 text write cells in both variants, against at most 0.19 for every other write cell; the earlier reports recorded the same spread for these cells. Write allocations differ between the variants by at most 22 B/op; the code is identical, so these differences are not attributed to the change.

Every candidate cell meets the tolerance on both profiles.

Possible regressions#

Two candidate-versus-baseline intervals outside the large reads lie entirely above 1, both on code that runs identically in both builds. Protobuf 3 nested serialize is at 1.274 (1.099, 1.477): four of the candidate’s five blocks ran at 0.91 to 0.92 times Chill’s time and one at 0.66, where all five baseline blocks ran at 0.67 to 0.68. Protobuf 3 text streamCopy, which never parses the payload, is at 1.042 (1.013, 1.072). As in the earlier reports, such intervals on unchanged paths are recorded as run-to-run drift between sequential runs and are not attributed to the change.

Limitations#

The two variants of each profile ran sequentially on one VM, over about two and a half hours, so host drift can affect native times; the Chill lane controls part of it within each block. Five blocks per run give limited information about the distribution of ratios. The plan is not compared with the earlier reports, which used other VMs, images, or JDK builds. The corpus has one large shape, a single 64 KiB bytes field; large frames dominated by proto3 strings, many small fields, unknown fields, or proto2 strings use the changed path but are not timed here, and the unit tests check only their correctness. Frames above 1 MiB, whose array grows by doubling, are tested for correctness but not timed. The benchmarks measure per-record time and allocation, not how long applications keep decoded values; the retention of frame arrays through kept ByteString values is a documented trade-off, not a measured cost. The corpus and warm setup do not cover first-use costs, other Flink or JDK versions, or record shapes outside the five workloads. Two earlier plans, issue113-read-1 and issue113-read-2, measured intermediate sources before the review changes, including one that read directly into the array that decoded values share; their results are not retained, and this report does not use them.

Retained evidence#

The data inventory lists the four result archives, their checksums, and how to reconstruct each measured tree.