feat: update metric agent to expose prometheus scrape metrics

[jeffreylimnardy]

What Changed

The metric agent OTel Collector now exposes Prometheus scrape health metrics (such as up, scrape_duration_seconds, and scrape_body_size_bytes) on a dedicated port so cluster operators can monitor scraping health without routing those metrics through user pipelines.

Affected Signal Types

Metrics

Key Changes

• internal/otelcollector/config/metricagent: Adds a new metrics/ops-scrape-metrics internal pipeline that keeps only the scrape health metric subset (opsScrapeMetricNames), drops healthy entries (where up == 1 and scrape_body_size_bytes > 0), and exports the remainder via a Prometheus exporter on port 9090. The pipeline is activated only when Prometheus or Istio input is enabled.
• internal/otelcollector/config/metricagent: Extends the enrichmentRoutingConnector to forward Prometheus- and Istio-sourced metrics to the new ops pipeline in addition to the per-user-pipeline outputs, using a new includeOpsScrapeMetrics flag.
• internal/otelcollector/config/metricagent: Enables extra_scrape_metrics: true on all Prometheus scrape jobs (pods, services, Istio proxy) and expands diagnosticMetricNames with scrape_timeout_seconds, scrape_sample_limit, and scrape_body_size_bytes.
• internal/otelcollector/config/common: Adds component ID constants ComponentIDOpsKeepScrapeMetricsProcessor, ComponentIDOpsDropHealthyScrapeMetricsProcessor, and ComponentIDOpsScrapeMetricsExporter, plus the PrometheusExporterConfig and ResourceToTelemetryConversion types.
• internal/otelcollector/ports: Adds the constant OpsScrapeMetrics = 9090 for the new Prometheus exporter endpoint.
• internal/resources/otelcollector: Opens port 9090 in the metric agent NetworkPolicy ingress rules and excludes port 9090 from Istio sidecar interception via the traffic.sidecar.istio.io/excludeInboundPorts annotation.

Notes for Reviewers

The ops pipeline drops metrics where up == 1 (healthy) and scrape_body_size_bytes > 0 before exporting, so only failed or anomalous scrape targets appear in the Prometheus endpoint by default. The scrape_timeout_seconds and scrape_sample_limit metrics are intentionally excluded from the ops pipeline (opsScrapeMetricNames) because they are static per-job values that add no per-target health signal. The ResourceToTelemetryConversion is enabled on the Prometheus exporter so resource attributes become Prometheus labels.

Release Notes Input

Recommended Action: If you scrape the metric agent's Prometheus endpoint (port 9090), you will now find scrape health metrics there for all active Prometheus and Istio scrape targets.

Metrics: The metric agent now exposes a Prometheus endpoint on port 9090 that reports scrape health metrics (up, scrape_duration_seconds, scrape_samples_scraped, scrape_samples_post_metric_relabeling, scrape_series_added, scrape_body_size_bytes) for Prometheus and Istio input sources. Only non-healthy entries (failed scrapes or zero-byte responses) are reported. This endpoint is active whenever at least one MetricPipeline has Prometheus or Istio input enabled.

• 🔄 Regenerate and Update Summary

PR Bot Information

Version: 1.23.0

• Event Trigger: issue_comment.edited
• Correlation ID: a5446bb3-d5eb-4356-8216-f455c4cec416

[hyperspace-pr-bot]

Control Panel

Hi, I'm an AI-powered Review Bot that helps you with summarizing and reviewing pull requests.
To interact with me, just use the following actions:

• 📝 Summarize PR
• 🔍 Review
• 🗑️ Delete all bot comments and reviews