Class PubSubMetricNames
Each connector has one of these, and comparing them is how the repository's metric naming
convention is held across connectors — a name that means the same thing in two connectors should
be spelled the same way, and a diff of these files is what shows it. The convention itself (a
counter names the event, a gauge names the state, and neither takes Flink's num prefix)
is recorded in the base module's detailed agent guidance.
What is not here: Flink's standard names, which come from
SinkWriterMetricGroup and SourceReaderMetricGroup accessors rather than from a name, and
the subgroup leaves base.metrics registers on this connector's behalf (
errorClass.CODE.errors, destination.TOPIC.recordsSend). The user-facing meaning of each
name is on the connector's documentation page, not duplicated here.
The dead-letter names below are this connector's and appear on somebody else's sink.
PubSubDeadLetterQueue serves every connector in this repository and registers on the
metric group FailureHandlerContext hands it — the host sink writer's — so a
BigQuery or Cloud Tasks job dead-lettering to a topic reports them beside that sink's own names.
They are declared here because the class is Pub/Sub's, the argument that already places its
options on reference/pubsub.md (ADR-0009), and each carries deadLetter so that it
reads unambiguously in that company.
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final Stringstatic final Stringstatic final StringBUFFERED_MESSAGESin bytes.static final StringWhat this subtask's subscribers are holding that the fetch loop has not taken yet — a state, so a gauge.static final Stringstatic final StringHow long the most recently completed wait for those publishes took, in milliseconds — the state to read againstflushTimeout, which is what a checkpoint's flush is spending.static final StringPUBLISHER_SHUTDOWNS_ABANDONEDfor the dead-letter publisher, counted separately and named separately because it is registered on a group that may already carry that name — the host sink's, when the host is a Pub/Sub sink.static final StringCounts dead letters the service has confirmed, at the point each publish future resolves, rather than at the offer that handed it to the client library.static final StringFETCHER_BUFFERED_MESSAGESin serialized bytes.static final StringMessages already removed from subscriber buffers but not yet taken from Flink's fetcher batches by the source reader.static final Stringstatic final Stringstatic final StringThe state the confirmed count is read against: dead letters handed to the client library and not yet resolved, whichmaxInFlightMessagesbounds.static final Stringstatic final StringThe longest of those waits so far, which is the one a reporter can actually catch (#405): waits happen as often as the queue drains — once per element underWRITE_THROUGH— so the last-wait gauge above is overwritten between two scrapes, and a publish that nearly spentflushTimeoutleaves no trace in it.static final Stringstatic final Stringstatic final Stringstatic final Stringstatic final Stringstatic final StringThe state a paused split is left in once its buffer outgrows its bound and the reader stops its subscriber (#357), so it takes the gauge shape — and "parked" in the sensePARKED_MESSAGESalready gives it here, held for a resumption that is expected.static final Stringstatic final Stringstatic final StringCounts an event — a publisher teardown the close gave up on — so it takes the counter shape the naming convention prescribes, even though its value comes from a process-wide total rather than this writer's own tally.static final StringThe sink writer counts a record its serializer skipped; the source reader counts a message its deserialization schema collected nothing for (ADR-0001).static final StringThe event behind that state, so it takes the counter shape.static final StringCounts the event a subscriber teardown is the only report of: a failure that reached the teardown having never been handed to the reader, so no job failure is coming for it (#351).static final StringPUBLISHER_SHUTDOWNS_ABANDONED's counterpart for the source's subscribers, spelled the same way because it means the same thing — one connector spelling one meaning twice is what this file exists to hold.static final Stringstatic final String -
Method Summary
-
Field Details
-
IN_FLIGHT_MESSAGES
- See Also:
-
IN_FLIGHT_BYTES
- See Also:
-
PARKED_MESSAGES
- See Also:
-
ACTIVE_PUBLISHERS
- See Also:
-
CAPACITY_EVICTIONS
- See Also:
-
IDLE_EVICTIONS
- See Also:
-
TOPICS_CREATED
- See Also:
-
PUBLISHER_SHUTDOWNS_ABANDONED
Counts an event — a publisher teardown the close gave up on — so it takes the counter shape the naming convention prescribes, even though its value comes from a process-wide total rather than this writer's own tally. The storage has to be process-wide (a per-attempt tally is unregistered before any reporter reads it, measured); the instrument does not follow from that, and a cumulative count of events is a counter. The count itself isPubSubShutdownResidue.- See Also:
-
RECORDS_SKIPPED
The sink writer counts a record its serializer skipped; the source reader counts a message its deserialization schema collected nothing for (ADR-0001). Separate groups, so the two series never collide, and separate counter implementations — the reader's is thread-safe and the writer's is not, for the reason each metrics class gives.- See Also:
-
MESSAGES_RECEIVED
- See Also:
-
MESSAGES_ACKED
- See Also:
-
MESSAGES_NACKED
- See Also:
-
MESSAGES_DROPPED
- See Also:
-
PENDING_ACKS
- See Also:
-
PENDING_CHECKPOINTS
- See Also:
-
BUFFERED_MESSAGES
What this subtask's subscribers are holding that the fetch loop has not taken yet — a state, so a gauge. It is the number the paused-split bound is evaluated against, and the memory a reader that has stopped draining accumulates whether or not any split is paused (#377).Two gauges rather than one, for the reason the bound they shadow has two dimensions: which of message count and byte size fills a TaskManager first depends on the message size.
Neither is
PENDING_ACKS, which counts messages received or emitted and not yet acknowledged and so cannot tell a growing buffer from a slow checkpoint. Nor do they cover the reader's whole footprint: everything already pulled has left the subscriber and left these, which is up to(source.reader.element.queue.capacity + 2) × maxRecordsPerFetch × splitsmessages — the queue itself, plus the fetch the reader is working through and the batch the fetcher cannot hand over, each of which holds one drain of every assigned split. Measured on #377 at 3999 against a capacity of 2, a 1000-message fetch and one split.- See Also:
-
BUFFERED_BYTES
BUFFERED_MESSAGESin bytes.- See Also:
-
FETCHER_BUFFERED_MESSAGES
Messages already removed from subscriber buffers but not yet taken from Flink's fetcher batches by the source reader. This includes a batch waiting to enter the element queue, the queue itself, and the batch the source reader is currently consuming.- See Also:
-
FETCHER_BUFFERED_BYTES
FETCHER_BUFFERED_MESSAGESin serialized bytes.- See Also:
-
PARKED_SPLITS
The state a paused split is left in once its buffer outgrows its bound and the reader stops its subscriber (#357), so it takes the gauge shape — and "parked" in the sensePARKED_MESSAGESalready gives it here, held for a resumption that is expected. This is the one to alert on: a split that stays parked is one an aligned group is holding indefinitely, which on a healthy job does not happen.- See Also:
-
SPLITS_PARKED
The event behind that state, so it takes the counter shape. Both exist because neither answers the other's question: a park and its resume falling between two scrapes leave the gauge at zero with nothing to say they happened, while the gauge alone cannot tell one long pause from an alignment cycle parking a split over and over.- See Also:
-
SUBSCRIBER_SHUTDOWNS_ABANDONED
PUBLISHER_SHUTDOWNS_ABANDONED's counterpart for the source's subscribers, spelled the same way because it means the same thing — one connector spelling one meaning twice is what this file exists to hold. Everything that field's javadoc says about the instrument and the process-wide storage holds here; the count itself isPubSubShutdownResidue.- See Also:
-
SUBSCRIBER_FAILURES_UNREPORTED
Counts the event a subscriber teardown is the only report of: a failure that reached the teardown having never been handed to the reader, so no job failure is coming for it (#351).Named for that property rather than for the shutdown, because the shutdown is where it is observed and not what it is: the same branch catches a streaming failure that landed after the reader's last pull, which no wording about shutting down would describe. What it excludes is the opposite case — the client repeating at teardown a failure the reader already has — which is counted by nothing, the job being already on its way down over that very failure. The count itself is
PubSubShutdownResidue.- See Also:
-
ASSIGNED_SPLITS
- See Also:
-
UNASSIGNED_READERS
- See Also:
-
DEAD_LETTERS_PUBLISHED
Counts dead letters the service has confirmed, at the point each publish future resolves, rather than at the offer that handed it to the client library. The offer-side count already exists on the same metric group and is not this connector's to register: every sink here incrementsnumRecordsSendErrorsimmediately before calling the failure handler, so undersendToDeadLetterQueue(...)it counts exactly what was offered. A hand-off counter here would be that series a second time, while what nothing reports is how much of it reached the topic.- See Also:
-
IN_FLIGHT_DEAD_LETTERS
The state the confirmed count is read against: dead letters handed to the client library and not yet resolved, whichmaxInFlightMessagesbounds.- See Also:
-
DEAD_LETTER_FLUSH_MILLIS
How long the most recently completed wait for those publishes took, in milliseconds — the state to read againstflushTimeout, which is what a checkpoint's flush is spending. Both waits the budget covers report here, the one inflush()and the onemaxInFlightMessagestriggers inside an offer.- See Also:
-
LONGEST_DEAD_LETTER_FLUSH_MILLIS
The longest of those waits so far, which is the one a reporter can actually catch (#405): waits happen as often as the queue drains — once per element underWRITE_THROUGH— so the last-wait gauge above is overwritten between two scrapes, and a publish that nearly spentflushTimeoutleaves no trace in it. Per task attempt, since it is writer state.- See Also:
-
DEAD_LETTER_PUBLISHER_SHUTDOWNS_ABANDONED
PUBLISHER_SHUTDOWNS_ABANDONEDfor the dead-letter publisher, counted separately and named separately because it is registered on a group that may already carry that name — the host sink's, when the host is a Pub/Sub sink. Flink resolves such a collision by keeping the metric registered first and dropping the other with a warning, so one name for both would make a healthy configuration log "Metric will not be reported". Splitting them also says which publisher is stalling. The count itself isPubSubShutdownResidue.- See Also:
-