Sizing traversal allocation in the native serializer#

This report measures the safe-sizing traversal investigated in #43 . After the buffer-allocation change , scalar and nested serialize and serializeSameInstance still took 6.2 to 7.0 times Chill’s time and allocated 1.9 to 2.2 KB/op; the 0.1.0 diagnostics had sampled allocation sites in the safe-sizing traversal. Two plans measure the change on temporary GCE VMs: one compares the final change with the unchanged serializer, and an earlier one separates the traversal bookkeeping, the descriptor work, and the remaining sizing cost from the output work with controlled variants. The measurement contract is ADR-0004 ; the comparison does not establish whole-job throughput or state compatibility.

Change under test#

Before the change, every write sized the record by collecting each message’s populated fields with getAllFields(), which builds a sorted map, and by allocating a traversal frame, a child list, an edge record per child, and a summary per completed value. A per-call identity map held every summary, including those of empty unknown-field sets, and each message asked its descriptor for its field list and per-field properties.

The changed sizing iterates over field layouts that each payload class computes once for every message type it reaches and caches with its other ClassValue metadata. A layout records each field’s type, tag size, repetition, packing, presence, whether reflection copies its list (maps and enums), whether it is a raw proto2 string, the required fields, and the layout of its message type. Traversal frames are cursors over those layouts; each serializer instance reuses up to 64 of them and clears their record references after every call. Within one call, only a subtree whose sizing examined more than 64 content units (declared fields, elements, unknown values, and string characters) is summarized by identity; smaller shared subtrees are sized again at each occurrence, which bounds that repeated work. A singular field without presence outside map entries, such as a proto3 scalar, is read once and compared with its default, which is how the runtime’s hasField decides its presence, instead of being read by both hasField and getField. Empty unknown-field sets and empty repeated enums are skipped, packed fixed-width fields are sized from their count, and packed varint fields of Internal.LongList or Internal.IntList are read without boxing. Checked arithmetic, size and depth limits, UTF-8 and raw proto2 string handling, unknown fields, and the exact-size check after writing are unchanged; ADR-0001 records the amended traversal.

ProtobufMessageSizeTest compares the calculated size with the runtime size for random graphs with shared children, fresh and memoized records, packed blocks with two- and three-byte length prefixes, implicit-presence fields at their defaults and at values that only resemble them (negative zero, NaN, and undeclared enum numbers), a map entry with a default key and value, shared unknown-field sets below and above the summary bound, and a value whose descriptor differs from the layout passed in. It also checks shared children above and below the summary bound and a summarized unknown-field set at deeper paths, rejects a 64-level doubling graph without exponential traversal, reuses one instance after failed checks, and confirms that no checked record stays reachable from the instance, including siblings pending in a repeated field, a map, or an unknown group when a check fails. The existing serializer tests continue to cover overflow above 4 GiB, depth limits through maps and unknown groups, raw strings, and inconsistent writers.

Method#

Both plans used the temporary GCE procedure in project flink-gcp. Each Protobuf profile ran on its own c3-standard-8 VM in asia-northeast1-b (Intel Xeon Platinum 8481C, Sapphire Rapids, 8 vCPUs, 32 GiB) from image ubuntu-2404-noble-amd64-v20261016, kernel 7.0.0-1014-gcp, with Temurin 17.0.20.1, Flink 2.3.0, and Protobuf/protoc 3.25.9 or 4.33.6. Every VM recorded TERMINATE host maintenance and NONE as the only maintenance event, passed the load gate before every build and measurement, and reported the same machine type, CPU platform, image, kernel, and JDK as the other VMs of its plan. just benchmark-pair ran the ADR-0004 long-run protocol for the native and Chill Protobuf lanes, as in the buffer-allocation report, and each variant took about 70 minutes.

In both plans, the baseline is the previous sizing and its tests, identical to the base under src/, and the candidate is the source commit unchanged. The candidate of issue43-sizing-2 is the change described above. The first candidate of issue43-sizing-1 lacks the single read of fields without presence and the count check of empty repeated enums; the variants that separate the sources are patches to it:

The floor is a diagnostic lower bound, not a supported configuration; it shows the output work that remains without safe sizing. The two plans are not compared with each other, because they ran at different times on different VMs.

Comparisons between variants of one plan use the Chill lane as a per-block control, as in the buffer-allocation report: the ratio of two variants’ geometric-mean native/Chill time ratios, with a pooled two-sample Student-t interval on the five log ratios of each run (8 degrees of freedom). Below 1 favors the first variant. Allocation is the five-fork mean of JMH’s normalized allocation, in bytes per operation. These intervals are per cell, without a simultaneous claim across cells.

Results#

The change against the previous sizing#

Each variant cell is native time in ns/op and allocation in B/op, as five-fork means, from plan issue43-sizing-2. The last three columns give the candidate’s time relative to the baseline and each variant’s native/Chill interval and ADR-0004 verdict.

Protobuf 4:

Protobuf 3:

The candidate takes 0.35 to 0.53 times the baseline’s time for scalar, nested, and collections writes on both profiles, 0.60 to 0.73 for text writes, and 0.85 to 0.94 for large writes, and every write interval lies below 1. It removes 1.6 to 1.9 KB/op from every scalar, text, nested, and large write and 11.1 to 12.4 KB/op from collections writes, leaving 272 to 357 B/op for previously encoded scalar and nested records and 600 to 666 B/op for previously encoded collections records. The reduction is similar in each size-cache state, so it does not come from generated size memoization; buildAndSerialize keeps the allocation of building the record. Text and large writes improve less in relative terms because UTF-8 counting or payload output dominates their time.

Small writes still miss the native/Chill tolerance, but closer: scalar and nested serialize and serializeSameInstance move from 6.2 to 7.1 times Chill’s time to 2.4 to 2.9 times, collections writes from 3.8 to 6.1 to 2.0 to 2.5, and buildAndSerialize of scalar and nested records from 3.1 to 3.3 to 1.5 to 1.6. Protobuf 4 text writes move from 1.8 to 2.2 to 1.3 to 1.6. The generated-size report measures the later change for these writes.

The only verdict change is Protobuf 3 text buildAndSerialize, from inconclusive to established.

Separating the sources#

Plan issue43-sizing-1 measured the first candidate with the descriptors and floor variants. The baseline-to-descriptors step measures the traversal bookkeeping and field enumeration, the descriptors-to-first-candidate step measures per-use descriptor work, and the floor-to-first-candidate step measures what safe sizing still cost.

Protobuf 4 write time and allocation:

Protobuf 4 decomposition:

Traversal bookkeeping and field enumeration are the larger source. On both profiles, the descriptors variant takes 0.52 to 0.67 times the baseline’s time for scalar, nested, and collections writes and removes 1.4 to 1.6 KB/op from scalar and nested writes and 9.4 to 10.9 KB/op from collections writes. Precomputed layouts then take a further 0.75 to 0.90 of the descriptors variant’s time for those writes and remove 120 to 144 B/op from scalar and nested writes and 1.3 to 1.5 KB/op from collections writes, whose map entries are messages with their own descriptors. Only Protobuf 4 is tabulated for this plan; the ranges in this section include the Protobuf 3 values, which can be computed from the retained measurements.

Safe sizing remained the largest part of a small write. The floor took 0.11 to 0.20 of the first candidate’s time for previously encoded scalar and nested records on both profiles and 0.32 to 0.36 for collections; between the first candidate and the floor, 224 to 574 B/op remained on every previously encoded record. That remainder includes Protobuf’s reflective field access, besides the traversal, depth checks, and UTF-8 counting; this report does not apportion it. In the pinned runtimes, hasField, getField, and getRepeatedFieldCount call generated accessors through Method.invoke, which boxes primitive results; map fields go through the generated map-field accessor instead, and map and repeated-enum getField copy their elements into a new list. The single read of fields without presence, added after this plan, removes one of those reflective calls for each such field with a non-default value; the plans are not compared, so its share of the difference between them is not measured. Removing reflection entirely would require reproducing protoc’s Java accessor names, which ADR-0001’s public-API design avoids. The floor met the native/Chill tolerance in 38 of 40 Protobuf 4 cells and 37 of 40 Protobuf 3 cells, missing only large deserialize and deserializeReuse and inconclusive for Protobuf 3 text serialize; the small-write gap to Chill is therefore safe sizing, not output or framing.

The first candidate’s verdicts matched the baseline’s except Protobuf 3 text serialize, from inconclusive to established. Intervals involving the Protobuf 3 text write cells of this plan are wide, for example 0.573 (0.374, 0.878) for the first candidate relative to the baseline. In the first-candidate and floor runs, the standard deviation of their five log native/Chill ratios is 0.27 to 0.54, against a median of about 0.02 for the other write cells; their native means still fall in the same variant order as on Protobuf 4. In the later plan, only the baseline’s Protobuf 3 text buildAndSerialize shows such a spread, at 0.34, which explains its wide interval and its inconclusive verdict.

Possible regressions#

Two of the 50 candidate-versus-baseline intervals outside the write operations in plan issue43-sizing-2, 25 per profile, lie entirely above 1: Protobuf 4 scalar streamCopy at 1.033 (1.009, 1.058) and Protobuf 3 text deserializeReuse at 1.010 (1.001, 1.019). In plan issue43-sizing-1, five such intervals lay above 1 for the first candidate, at most 1.049 (1.031, 1.068) for Protobuf 3 nested streamCopy; none of them recurs in the later plan. Reading and stream copy run identical code in both builds; the only other difference is that the first metadata resolution for each class also builds its layouts. The buffer-allocation report found the same pattern of small, inconsistent intervals on unchanged paths, so these intervals are recorded as run-to-run drift between sequential runs and are not attributed to the change.

Limitations#

The variants of each plan ran sequentially on one VM per profile, over about two and a half hours in the later plan and five in the earlier one, so host drift can affect native times; the Chill lane controls part of it within each block. Five blocks per run give limited information about the distribution of ratios. Neither plan is compared with the buffer-allocation report, which used a different VM, image, and kernel. The corpus has no shared children, deep nesting, unknown fields, enum fields, or messages with many declared fields; the summary bound and the release of frames beyond the 64 retained affect those shapes, which the unit tests check for correctness but these measurements do not time. The corpus and warm setup do not cover first-use costs, other Flink or JDK versions, or record shapes outside the five workloads.

Retained evidence#

The data inventory lists the twelve result archives of both plans, their checksums, and how to reconstruct each measured tree.