OpenTelemetry Support

Overview

The io.Manager Server can be configured to export metrics via OpenTelemetry.

By default, the io.Manager Server supports exporting metrics to an OpenTelemetry Collector. It's also possible to provide your own custom metrics exporter and metrics reader pointing to a different backend service.

The io.Manager Server publishes a set of metrics which you can enable or disable and configure individually.

Enabling Metrics Exports

Publishing metrics data is disabled by default. To enable publishing metrics, you must enable OpenTelemetry support in the io.Manager Server and explicitly enable the metrics feature via the configuration object for initializing the io.Manager Server, or via environment variables, depending on your deployment approach.

Environment Variables

To enable and configure publishing metrics, register the following environment variables with the proper values. The API_OTEL_METRICS_ENABLED environment variable must be set to true:

Environment Variable Description
API_OTEL_METRICS_DEFAULT_METRICS_ENABLED If true (default), will enable publishing of all available metrics.
API_OTEL_METRICS_ENABLED If true, will enable publishing OpenTelemetry metrics. Defaults to false.
API_OTEL_METRICS_PUBLISH_INTERVAL Interval in milliseconds at which the metrics reader will collect metrics. Passed as a value to the exportIntervalMillis property of the PeriodicExportingMetricReaderOptions constructor parameter when creating a PeriodicExportingMetricReader instance. This is the default metrics reader used by the io.Manager Server. Ignored when a custom metrics reader is used. Defaults to 5000.
API_OTEL_METRICS_URL URL pointing to an OpenTelemetry Collector where the generated metrics will be sent via HTTP. Passed as a value to the url property of the OTLPExporterNodeConfigBase constructor parameter when creating an OTLPMetricExporter instance. This is the default metrics exporter used by the io.Manager Server. Required if using the default metrics exporter and metrics reader. Ignored when either a custom metrics exporter or a custom metrics reader is used.

The following example demonstrates how to enable OpenTelemetry support, how to enable publishing metrics, and how to configure the interval for publishing metrics:

# Enabling OpenTelemetry support.
API_OTEL_ENABLED=true
API_OTEL_RESOURCE_SERVICE_NAME=io-manager

# Enabling and configuring metrics.
API_OTEL_METRICS_ENABLED=true
API_OTEL_METRICS_URL=http://localhost:4318/v1/metrics
API_OTEL_METRICS_PUBLISH_INTERVAL=10000

Configuration Object

To enable publishing metrics, use the metrics property under the otel top-level key of the optional Config object for initializing the io.Manager Server.

The following example demonstrates how to enable OpenTelemetry support, how to enable publishing metrics, and how to configure the interval for publishing metrics:

import { start } from "@interopio/manager";

const config = {
    // Enabling OpenTelemetry support.
    otel: {
        enabled: true,
        resource: {
            serviceName: "io-manager"
        },
        // Enabling and configuring metrics.
        metrics: {
            enabled: true,
            url: "http://localhost:4318/v1/metrics",
            publishInterval: 10000
        }
    }
};

const server = await start(config);

The metrics object has the following properties:

Property Type Description
customExporter object PushMetricExporter instance of a custom metrics exporter to be used instead of the default metrics exporter. Ignored when a custom metrics reader is used.
customReader object MetricReader instance of a custom metrics reader to be used instead of the default metrics reader.
defaultMetricsEnabled boolean If true (default), will enable publishing of all available metrics.
enabled boolean If true, will enable publishing OpenTelemetry metrics. Defaults to false.
metrics object[] A list of metric definitions used to provide configuration for supported metrics.
publishInterval number Interval in milliseconds at which the metrics reader will collect metrics. Passed as a value to the exportIntervalMillis constructor parameter when creating a PeriodicExportingMetricReader instance. This is the default metrics reader used by the io.Manager Server. Ignored when a custom metrics reader is used. Defaults to 5000.
url string URL pointing to an OpenTelemetry Collector where the generated metrics will be sent via HTTP. Passed as a value to the url constructor parameter when creating an OTLPMetricExporter instance. This is the default metrics exporter used by the io.Manager Server. Required if using the default metrics exporter and metrics reader. Ignored when either a custom metrics exporter or a custom metrics reader is used.

Available Metrics

The io.Manager Server exports the following metrics:

Metric Instrument Type Description
"http.server.active_requests" UpDownCounter Number of active HTTP server requests.
"http.server.request.body.size" Histogram Size of HTTP server request bodies.
"http.server.request.duration" Histogram Duration of HTTP server requests.
"http.server.response.body.size" Histogram Size of HTTP server response bodies.
"io_manager.active_sessions" Gauge Total number of sessions that have made requests within a past interval. This interval is defined by the inactiveSessionTimeout metric property, it's measured in seconds and is configurable.

All available metrics are enabled by default. It's possible to disable all available metrics and explicitly enable and configure individual ones via the configuration object for initializing the io.Manager Server, or via environment variables, depending on your deployment approach. Each metric has specific configurable properties (for more details about each individual metric, see the respective metric section).

To disable all available metrics via environment variables, set the API_OTEL_METRICS_DEFAULT_METRICS_ENABLED variable to false. To explicitly enable and configure individual metrics, use the respective environment variables listed in the sections describing the individual metrics.

The following example demonstrates how to disable all available metrics and explicitly enable and configure the "io_manager.active_sessions" metric via environment variables:

# Enabling OpenTelemetry support.
API_OTEL_ENABLED=true
API_OTEL_RESOURCE_SERVICE_NAME=io-manager

# Enabling metrics.
API_OTEL_METRICS_ENABLED=true
API_OTEL_METRICS_URL=http://localhost:4318/v1/metrics

# Disabling all available metrics.
API_OTEL_METRICS_DEFAULT_METRICS_ENABLED=false

# Enabling and configuring the `"io_manager.active_sessions"` metric.
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_ENABLED=true
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_PUBLISH_INTERVAL=45000

To disable all available metrics via the configuration object for initializing the io.Manager Server, set the defaultMetricsEnabled property of the metrics object to false. To explicitly enable and configure individual metrics, use the metrics property of the metrics object. It accepts a list of metrics definition objects. To specify the metric you want to enable and configure, provide the metric name as a value to the type property of the metrics definition object.

The following example demonstrates how to disable all available metrics and explicitly enable and configure the "io_manager.active_sessions" metric:

import { start } from "@interopio/manager";

const config = {
    otel: {
        enabled: true,
        resource: {
            serviceName: "io-manager"
        },
        metrics: {
            enabled: true,
            url: "http://localhost:4318/v1/metrics",
            // Disabling all available metrics
            defaultMetricsEnabled: false,
            // Enabling and configuring the `"io_manager.active_sessions"` metric.
            metrics: [
                {
                    type: "io_manager.active_sessions",
                    enabled: true,
                    publishInterval: 45000
                }
            ]
        }
    }
};

const server = await start(config);

The following sections describe each individual metric published by the io.Manager Server.

http.server.active_requests

The instrument type of the "http.server.active_requests" metric is UpDownCounter.

The following environment variables are available for configuring the "http.server.active_requests" metric:

Environment Variable Description
API_OTEL_METRICS_DEFINITIONS_HTTP_ACTIVE_REQUESTS_DESCRIPTION Description for the metric. Defaults to "Number of active HTTP server requests.".
API_OTEL_METRICS_DEFINITIONS_HTTP_ACTIVE_REQUESTS_ENABLED If true, will enable publishing the metric. Defaults to the value of the API_OTEL_METRICS_DEFAULT_METRICS_ENABLED environment variable.
API_OTEL_METRICS_DEFINITIONS_HTTP_ACTIVE_REQUESTS_NAME Name for the metric. May be used in visualization tools. Defaults to http.server.active_requests.
API_OTEL_METRICS_DEFINITIONS_HTTP_ACTIVE_REQUESTS_UNIT Unit for the metrics values. Defaults to {request}.

The following properties are available for configuring the "http.server.active_requests" metric via the configuration object for initializing the io.Manager Server:

Property Type Description
description string Description for the metric. Defaults to "Number of active HTTP server requests.".
enabled boolean If true, will enable publishing the metric. Defaults to the value of the defaultMetricsEnabled property of the metrics object.
name string Name for the metric. May be used in visualization tools. Defaults to "http.server.active_requests".
unit string Unit for the metrics values. Defaults to "{request}".

http.server.request.body.size

The instrument type of the "http.server.request.body.size" metric is Histogram.

The following environment variables are available for configuring the "http.server.request.body.size" metric:

Environment Variable Description
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_BODY_SIZE_BUCKETS Histogram bucket boundaries. Defaults to [1024, 10240, 102400, 1048576, 10485760, 104857600].
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_BODY_SIZE_DESCRIPTION Description for the metric. Defaults to "Size of HTTP server request bodies.".
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_BODY_SIZE_ENABLED If true, will enable publishing the metric. Defaults to the value of the API_OTEL_METRICS_DEFAULT_METRICS_ENABLED environment variable.
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_BODY_SIZE_NAME Name for the metric. May be used in visualization tools. Defaults to http.server.request.body.size.
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_BODY_SIZE_UNIT Unit for the metrics values. Defaults to By.

The following properties are available for configuring the "http.server.request.body.size" metric via the configuration object for initializing the io.Manager Server:

Property Type Description
buckets number[] Histogram bucket boundaries. Defaults to [1024, 10240, 102400, 1048576, 10485760, 104857600].
description string Description for the metric. Defaults to "Size of HTTP server request bodies.".
enabled boolean If true, will enable publishing the metric. Defaults to the value of the defaultMetricsEnabled property of the metrics object.
name string Name for the metric. May be used in visualization tools. Defaults to "http.server.request.body.size".
unit string Unit for the metrics values. Defaults to "By".

http.server.request.duration

The instrument type of the "http.server.request.duration" metric is Histogram.

The following environment variables are available for configuring the "http.server.request.duration" metric:

Environment Variable Description
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_BUCKETS Histogram bucket boundaries. Defaults to [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10].
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_DESCRIPTION Description for the metric. Defaults to "Duration of HTTP server requests.".
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_ENABLED If true, will enable publishing the metric. Defaults to the value of the API_OTEL_METRICS_DEFAULT_METRICS_ENABLED environment variable.
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_NAME Name for the metric. May be used in visualization tools. Defaults to http.server.request.duration.
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_UNIT Unit for the metrics values. Defaults to s.

The following properties are available for configuring the "http.server.request.duration" metric via the configuration object for initializing the io.Manager Server:

Property Type Description
buckets number[] Histogram bucket boundaries. Defaults to [0.005, 0.01, 0.025, 0.05, 0.075, 0.1, 0.25, 0.5, 0.75, 1, 2.5, 5, 7.5, 10].
description string Description for the metric. Defaults to "Duration of HTTP server requests.".
enabled boolean If true, will enable publishing the metric. Defaults to the value of the defaultMetricsEnabled property of the metrics object.
name string Name for the metric. May be used in visualization tools. Defaults to "http.server.request.duration".
unit string Unit for the metrics values. Defaults to "s".

http.server.response.body.size

The instrument type of the "http.server.response.body.size" metric is Histogram.

The following environment variables are available for configuring the "http.server.response.body.size" metric:

Environment Variable Description
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_BUCKETS Histogram bucket boundaries. Defaults to [1024, 10240, 102400, 1048576, 10485760, 104857600].
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_DESCRIPTION Description for the metric. Defaults to "Size of HTTP server response bodies.".
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_ENABLED If true, will enable publishing the metric. Defaults to the value of the API_OTEL_METRICS_DEFAULT_METRICS_ENABLED environment variable.
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_NAME Name for the metric. May be used in visualization tools. Defaults to http.server.response.body.size.
API_OTEL_METRICS_DEFINITIONS_HTTP_REQUEST_DURATION_UNIT Unit for the metrics values. Defaults to By.

The following properties are available for configuring the "http.server.response.body.size" metric via the configuration object for initializing the io.Manager Server:

Property Type Description
buckets number[] Histogram bucket boundaries. Defaults to [1024, 10240, 102400, 1048576, 10485760, 104857600].
description string Description for the metric. Defaults to "Size of HTTP server response bodies.".
enabled boolean If true, will enable publishing the metric. Defaults to the value of the defaultMetricsEnabled property of the metrics object.
name string Name for the metric. May be used in visualization tools. Defaults to "http.server.response.body.size".
unit string Unit for the metrics values. Defaults to "By".

io_manager.active_sessions

The instrument type of the "io_manager.active_sessions" metric is Gauge.

The following environment variables are available for configuring the "io_manager.active_sessions" metric:

Environment Variable Description
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_DESCRIPTION Description for the metric. Defaults to "Size of HTTP server response bodies.".
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_ENABLED If true, will enable publishing the metric. Defaults to the value of the API_OTEL_METRICS_DEFAULT_METRICS_ENABLED environment variable.
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_INACTIVE_SESSION_TIMEOUT Interval in seconds after which a session is considered inactive. Must be set to the same value as the fetchInterval property of the server object in the platform configuration (io.Connect Desktop or io.Connect Browser) for io.Manager. Defaults to 30.
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_NAME Name for the metric. May be used in visualization tools. Defaults to io_manager.active_sessions.
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_PUBLISH_AT_STARTUP_ENABLED If true (default), the io.Manager Server will publish the metric at startup.
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_PUBLISH_INTERVAL Interval in milliseconds at which to publish the metric. Defaults to 60000.
API_OTEL_METRICS_DEFINITIONS_ACTIVE_SESSIONS_UNIT Unit for the metrics values. Defaults to {session}.

The following properties are available for configuring the "io_manager.active_sessions" metric via the configuration object for initializing the io.Manager Server:

Property Type Description
description string Description for the metric. Defaults to "Total number of sessions that made requests in the past `inactiveSessionTimeout` seconds.".
enabled boolean If true, will enable publishing the metric. Defaults to the value of the defaultMetricsEnabled property of the metrics object.
inactiveSessionTimeout number Interval in seconds after which a session is considered inactive. Must be set to the same value as the fetchInterval property of the server object in the platform configuration (io.Connect Desktop or io.Connect Browser) for io.Manager. Defaults to 30.
name string Name for the metric. May be used in visualization tools. Defaults to "io_manager.active_sessions".
publishAtStartupEnabled boolean If true (default), the io.Manager Server will publish the metric at startup.
publishInterval number Interval in milliseconds at which to publish the metric. Defaults to 60000.
unit string Unit for the metrics values. Defaults to "{session}".

Customization

Metrics Exporter

The io.Manager Server provides an out-of-the-box implementation of an OTLPMetricExporter for sending metrics to an OpenTelemetry Collector via HTTP.

For more advanced scenarios, it's possible to provide an instance of a custom metrics exporter by using the customExporter property of the metrics object in the configuration object for initializing the io.Manager Server.

⚠️ Note that it isn't possible to provide a custom metrics exporter via environment variables.

For a complete example, see the OpenTelemetry Custom Metrics Exporter example on GitHub.

Metrics Reader

The io.Manager Server provides an out-of-the-box implementation of a PeriodicExportingMetricReader for exporting metrics periodically.

For more advanced scenarios, it's possible to provide an instance of a custom metrics reader by using the customReader property of the metrics object in the configuration object for initializing the io.Manager Server.

⚠️ Note that it isn't possible to provide a custom metrics reader via environment variables.

For a complete example, see the OpenTelemetry Custom Metrics Reader example on GitHub.