Package io.github.flink.gcp.connector.base.metrics

Class ClientMetrics

java.lang.Object
io.github.flink.gcp.connector.base.metrics.ClientMetrics
All Implemented Interfaces:
AutoCloseable
@Internal public final class ClientMetrics extends Object implements AutoCloseable
Turns a Google Cloud client library's own OpenTelemetry metrics into Flink metrics, so that every configured Flink metric reporter sees them.

A connector hands openTelemetry() to the client it builds, through the client's metrics-provider hook, and registers one Flink Counter per series it wants with counterSum(String, String), histogramCount(String, String) or histogramSumMillis(String, String). The instance implements the part of the OpenTelemetry Metrics API those clients record through, and nothing more: no OpenTelemetry SDK is involved, and nothing touches GlobalOpenTelemetry, so a shaded copy stays self-contained.

Recording

A client's add or record adds straight into the accumulator of the series its instrument and its method attribute name, on the client's own thread. A Flink counter reads that accumulator, so a reporter always reads the current total, and this class owns no thread, buffer or collection step. Every attribute other than the method is ignored, so a component reports instruments times methods series whatever the client labels its data with.

What is kept, per instrument:

  • a long counter: the total of what was added;
  • a histogram, double or long: how many values were recorded and, for a duration in ms or s, their sum in milliseconds. Under any other unit only the count is kept. Percentiles, buckets and extremes are not kept: a backend computes the mean over any window as rate(sum) / rate(count).

Every counter only grows. A negative or non-finite value is dropped, as the OpenTelemetry SDK drops it, because a reporter's backend computes rates from these counters. A count and its sum are two accumulators, so a reporter reading both while the client records may see one more value in one than in the other; over any rate window that difference vanishes.

What is not reported

Up-down counters, gauges, double counters and every observable (callback) instrument are handed to OpenTelemetry's no-op meter, with one WARN per instrument name. So is a recording for a method no counter was registered for, with one WARN per method, and a recording into an instrument nothing was registered for, with one per instrument, unless the connector declared it with ignore(String).

Registration happens at open, not on first sight

The connector registers every counter when its component opens, for the RPC methods that component's client calls. A registration made later, from a client's thread, would run while a reporter iterates its own metric map: Slf4jReporter iterates without a lock and loses that report to the ConcurrentModificationException.

Close the client before this object. close() stops it accepting recordings, so the counters keep their last values rather than reading zero.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
    Stops accepting recordings; every counter keeps the value it has.
    org.apache.flink.metrics.Counter
    counterSum(String instrument, String method)
    Returns a counter reading the total of a long counter instrument for one method.
    static ClientMetrics
    create(String methodAttribute)
    Creates the client metrics of one component, whose series are keyed by the given method attribute.
    org.apache.flink.metrics.Counter
    histogramCount(String instrument, String method)
    Returns a counter reading how many values a histogram instrument recorded for one method.
    org.apache.flink.metrics.Counter
    histogramSumMillis(String instrument, String method)
    Returns a counter reading the sum of a duration histogram for one method, in milliseconds, rounded to the nearest millisecond.
    void
    ignore(String instrument)
    Drops an instrument the connector deliberately does not report, without the warning an unregistered instrument gets.
    io.opentelemetry.api.OpenTelemetry
    openTelemetry()
    Returns the instance to hand to the client's metrics-provider hook.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • create

      public static ClientMetrics create(String methodAttribute)
      Creates the client metrics of one component, whose series are keyed by the given method attribute.
      Parameters:
      methodAttribute - the attribute the client labels each RPC method with, the one attribute kept
      Returns:
      the client metrics, owned by the caller

    • openTelemetry

      public io.opentelemetry.api.OpenTelemetry openTelemetry()
      Returns the instance to hand to the client's metrics-provider hook. It carries this object's meter provider and no-op tracing and propagation.

    • counterSum

      public org.apache.flink.metrics.Counter counterSum(String instrument, String method)
      Returns a counter reading the total of a long counter instrument for one method.
      Parameters:
      instrument - the instrument name, as the client creates it
      method - the method attribute's value
      Returns:
      a read-only counter to register on a metric group

    • histogramCount

      public org.apache.flink.metrics.Counter histogramCount(String instrument, String method)
      Returns a counter reading how many values a histogram instrument recorded for one method.
      Parameters:
      instrument - the instrument name, as the client creates it
      method - the method attribute's value
      Returns:
      a read-only counter to register on a metric group

    • histogramSumMillis

      public org.apache.flink.metrics.Counter histogramSumMillis(String instrument, String method)
      Returns a counter reading the sum of a duration histogram for one method, in milliseconds, rounded to the nearest millisecond. The instrument's unit must be ms or s; under any other unit the counter reads zero and it logs the unit once, because a number in an unknown unit under a Millis name would be false.
      Parameters:
      instrument - the instrument name, as the client creates it
      method - the method attribute's value
      Returns:
      a read-only counter to register on a metric group

    • ignore

      public void ignore(String instrument)
      Drops an instrument the connector deliberately does not report, without the warning an unregistered instrument gets. Use it for an instrument known to record something wrong, and say why where it is called.
      Parameters:
      instrument - the instrument name, as the client creates it

    • close

      public void close()
      Stops accepting recordings; every counter keeps the value it has.
      Specified by:
      close in interface AutoCloseable