Prometheus Plugin

TLS settings location

TLS modes, CA/client certificates and keys, server names and verification options are grouped under Security > TLS. Endpoint/port fields keep their original location. Grouping does not change field keys, saved values, defaults, sensitive handling, visibility or validation.

Basic accessibility observation

Resource overviews show accessible, inaccessible, or access unknown independently of metric sources. Successful access does not attest healthy membership, replication, partitions, or workloads; use domain diagnostics and alerts for those conditions. Existing liveness/readiness fields are domain evidence and do not drive the single core-owned access alarm. Unknown observations do not resolve firing incidents.

TLS

Enable tls_insecure_skip_verify in connection settings to retain TLS encryption without verifying the server certificate. It defaults to false and does not require a CA when enabled. Mutual TLS still requires a client certificate and key when requested by the server. Keep CA-based verification enabled when server identity must be authenticated.

Resource Metrics settings support named Mapping Profiles. Resources sharing a source and profile share corrections; a different name isolates exporter mappings. Empty retains the existing default profile. Diagnosis checks instant and range queries separately and distinguishes no data, query failures and invalid mappings. Counter-name discovery runs during validation/inspection. Runtime queries reuse cached selector choices; revalidate after restart or eviction to warm them again. Automatic mapping-pack application compiles the queries during resource registration without querying Prometheus. Run mapping diagnosis or manual preview to check actual data and warm counter-name discovery.

Counter metric names can be resolved automatically with or without _total. The configured name wins when it exists in the resource's label/time scope. Otherwise, counter metadata with consistent units and scoped discovery must confirm the alternative. Missing metadata, conflicting types, and partial discovery leave the mapping unchanged. Prefix settings apply first. Inspect the query to see the selected name. Histogram suffixes and unit conversions still require explicit mappings; offset/@ selectors are not rewritten.

Target health shows the observed up ratio and down-target count in compact cards. The distribution pie remains available for personalization but is not a default panel. Per-job targets retain a full-width comparison; collection and rule/alert trends use three columns and query/storage trends four.

Empty target observations and query failures remain distinct from a healthy zero. Saved layouts, query scope and budgets are unchanged.

Counts and alerts are displayed as integers; scaled storage bytes, percentages, rates, and durations keep useful fractional digits. Job and target-state distributions add their component series, while already aggregated queries remain single latest-value summaries. Missing data still differs from an observed zero.

The Prometheus plugin is a metric source plugin that lets Konduo query Prometheus-compatible HTTP APIs and execute resource-owned logical metrics as PromQL queries.

Highlights

• Executes range and instant queries for resource dashboards and alert evaluation.
• Supports label exploration, query inspection, and mapping-pack review.
• Can register a Prometheus server itself as a resource to inspect targets, TSDB, rule/evaluation, and runtime signals.
• Protects management actions such as reload and snapshot behind configuration, confirmation, and cooldown controls.

Before Registration

• The Konduo backend must be able to reach the Prometheus HTTP API.
• Prepare endpoint URL, authentication mode, timeout, and query-limit policy.
• If this Prometheus instance will be used as a managed metric collection write target, run it with --web.enable-remote-write-receiver.
• Apply resource plugin mapping packs so dashboards and alert rules can resolve expected logical metrics.

Operations Tips

Use Existing Metric Name Prefix and New Metric Name Prefix in the resource's Metrics settings. Leave the existing prefix empty and enter prefix_ as the new prefix to prepend it to every mapped name. Set existing pg_ and new prefix_ to query prefix_up instead of pg_up; unmatched names stay unchanged. An empty new prefix removes the existing prefix; both empty keep original names. Dashboards, alerts, inspection, and diagnostic evidence share these settings. Keep resource label filters; other exporter name or unit differences still need mapping corrections. Removal that produces an invalid metric name is rejected.

• If dashboards are empty, check scrape state, label filters, and mapping-pack installation together.
• Bound query ranges and steps to avoid excessive sample volume.
• Alert query batches use four concurrent Prometheus requests by default. Set metric_query_batch_concurrency to 1 for sequential execution when the source returns 429 responses or shows query pressure; values above 16 are clamped. Concurrency reduces waiting time but does not reduce request count.
• Bucket min/max exploration evaluates simple series selectors over all stored samples. Derived expressions use the requested sample step; spikes or dips shorter than that interval can be missed, so a sampled result must not be interpreted as exhaustive absence across the range.
• Treat Prometheus target health as metric evidence, not as the direct availability source for the scraped resource.
• When a supported query succeeds and ALERTS{alertstate="firing"} returns no event series in its requested scope, stat, gauge and distribution panels show a check and No active alerts.
• Timelines keep the selected time axis with a faint dashed guide and a small No events observed in this range caption, including when expanded. Empty timelines can be expanded; the guide is not a measured zero. Actual zero samples remain real charts.
• Query information explains the query mode and range. An instant value does not establish absence throughout the selected range or confirm healthy collection/resources. Missing-data and query-error behavior remains unchanged; a failed or unavailable query remains unknown.

Operator Views

• The metric source view is the entry point for HTTP API connectivity, label exploration, query inspection, and mapping-pack status.
• The Prometheus resource view exposes Prometheus' own targets, rule groups, alert state, TSDB, and runtime status.
• Target health uses a compact up-ratio headline, an up/down count distribution, and a Job breakdown so each panel answers a distinct triage question.
• The default dashboard adds time series for accepted samples per scrape, query evaluation duration, TSDB block/WAL storage, and rule evaluation duration alongside availability and throughput.
• Query and rule evaluation duration are five-minute averages. Compare them with query rate and rule intervals, and review TSDB storage together with retention and compaction health.
• If a Prometheus-compatible backend does not expose Prometheus's own metric names, these optional panels can show no data without failing mapping-pack validation.
• Mapping-pack views show how a resource plugin's logical metric keys resolve to PromQL.
• Query inspection helps diagnose empty dashboards or partial alert evaluation by showing the actual query and label matchers.

Diagnostics And Alerts

• Because this plugin provides metric evidence for other resources, target health, scrape delay, and rule evaluation state are operationally important.
• Diagnostics separate HTTP API connectivity, scrape state, rule evaluation, active alerts, TSDB/runtime pressure, and mapping coverage.
• Prometheus alerting covers metric-platform risk; resource outage decisions for Redis, Kafka, PostgreSQL, and others remain owned by the relevant plugin and Core availability.
• When queries are slow or sample volume is high, tune range, step, label filters, and dashboard panel count together.

Management Boundary

• Reload and snapshot are operationally meaningful management actions and stay behind confirmation and cooldown controls.
• Arbitrary PromQL query support is for operational inspection, not a general long-running analytics workbench.
• When Prometheus is used as a managed collection write target, it stores samples; resource plugins and Core own collection cadence and target access.
• Mapping-pack overrides adapt exporter metric names and labels; they must not be used to mask real resource health.

Example Workflows

• For an empty resource dashboard, check Prometheus target health, label filters, mapping-pack application, and query-inspect output in order.
• For partial alert evaluation, compare the logical metric's PromQL with actual series labels.
• For Prometheus pressure, review rule evaluation delay, query concurrency, TSDB head block behavior, and scrape intervals.
• If managed collection samples are missing, confirm remote-write receiver mode, Core write status, and the resource identity label.

---

Prometheus Enterprise Overlay

TLS settings location

TLS modes, CA/client certificates and keys, server names and verification options are grouped under Security > TLS. Endpoint/port fields keep their original location. Grouping does not change field keys, saved values, defaults, sensitive handling, visibility or validation.

Basic accessibility observation

Resource overviews show accessible, inaccessible, or access unknown independently of metric sources. Successful access does not attest healthy membership, replication, partitions, or workloads; use domain diagnostics and alerts for those conditions. Existing liveness/readiness fields are domain evidence and do not drive the single core-owned access alarm. Unknown observations do not resolve firing incidents.

TLS

Enable tls_insecure_skip_verify in connection settings to retain TLS encryption without verifying the server certificate. It defaults to false and does not require a CA when enabled. Mutual TLS still requires a client certificate and key when requested by the server. Keep CA-based verification enabled when server identity must be authenticated.

Resource Metrics settings support named Mapping Profiles. Resources sharing a source and profile share corrections; a different name isolates exporter mappings. Empty retains the existing default profile. Diagnosis checks instant and range queries separately and distinguishes no data, query failures and invalid mappings. Counter-name discovery runs during validation/inspection. Runtime queries reuse cached selector choices; revalidate after restart or eviction to warm them again. Automatic mapping-pack application compiles the queries during resource registration without querying Prometheus. Run mapping diagnosis or manual preview to check actual data and warm counter-name discovery.

Counter metric names can be resolved automatically with or without _total. The configured name wins when it exists in the resource's label/time scope. Otherwise, counter metadata with consistent units and scoped discovery must confirm the alternative. Missing metadata, conflicting types, and partial discovery leave the mapping unchanged. Prefix settings apply first. Inspect the query to see the selected name. Histogram suffixes and unit conversions still require explicit mappings; offset/@ selectors are not rewritten.

Target health shows the observed up ratio and down-target count in compact cards. The distribution pie remains available for personalization but is not a default panel. Per-job targets retain a full-width comparison; collection and rule/alert trends use three columns and query/storage trends four.

Empty target observations and query failures remain distinct from a healthy zero. Saved layouts, query scope and budgets are unchanged.

Counts and alerts are displayed as integers; scaled storage bytes, percentages, rates, and durations keep useful fractional digits. Job and target-state distributions add their component series, while already aggregated queries remain single latest-value summaries. Missing data still differs from an observed zero.

The Prometheus Enterprise overlay adds MCP descriptors and Prometheus self-telemetry anomaly rules to metric source resources. It detects metric platform risk through scrape, ingestion, cardinality, and query pressure signals.

Metric name prefixes

Use Existing Metric Name Prefix and New Metric Name Prefix in the resource's Metrics settings. Leave the existing prefix empty and enter prefix_ as the new prefix to prepend it to every mapped name. Set existing pg_ and new prefix_ to query prefix_up instead of pg_up; unmatched names stay unchanged. An empty new prefix removes the existing prefix; both empty keep original names. Dashboards, alerts, inspection, and diagnostic evidence share these settings. Keep resource label filters; other exporter name or unit differences still need mapping corrections. Removal that produces an invalid metric name is rejected.

Metric interpretation

• Target health uses a compact up-ratio headline, an up/down count distribution, and a Job breakdown so each panel answers a distinct triage question.

Highlights

• Exposes Prometheus resource metadata, diagnostics, metrics catalog, and anomaly rules through MCP.
• Declarative metric rules use logical metrics such as scrape duration, ingestion rate, TSDB head series, and query rate.
• Simulator series validate cardinality and ingestion pressure scenarios.
• Prometheus remains a metric provider and does not run plugin-mode detectors.

Before Registration

• Prometheus API endpoint and query permissions must be healthy.
• Connect a mapping pack for Prometheus self-metrics.
• MCP users need metric-source read permission and the required scopes.

Operations Tips

• Track cardinality growth together with exporter, label, and instrumentation changes.
• When scrape latency and ingestion pressure rise together, inspect target growth and remote write state.
• Alert query batches use four concurrent Prometheus requests by default. Set metric_query_batch_concurrency to 1 for sequential execution when the source returns 429 responses or shows query pressure; values above 16 are clamped. Concurrency reduces waiting time but does not reduce request count.
• Bucket min/max exploration evaluates simple series selectors over all stored samples. Derived expressions use the requested sample step; spikes or dips shorter than that interval can be missed, so a sampled result must not be interpreted as exhaustive absence across the range.
• When a supported query succeeds and ALERTS{alertstate="firing"} returns no event series in its requested scope, stat, gauge and distribution panels show a check and No active alerts.
• Timelines keep the selected time axis with a faint dashed guide and a small No events observed in this range caption, including when expanded. Empty timelines can be expanded; the guide is not a measured zero. Actual zero samples remain real charts.
• Query information explains the query mode and range. An instant value does not establish absence throughout the selected range or confirm healthy collection/resources. Missing-data and query-error behavior remains unchanged; a failed or unavailable query remains unknown.
• The default dashboard adds time series for accepted samples per scrape, query evaluation duration, TSDB block/WAL storage, and rule evaluation duration alongside availability and throughput.
• Query and rule evaluation duration are five-minute averages. Compare them with query rate and rule intervals, and review TSDB storage together with retention and compaction health.
• If a Prometheus-compatible backend does not expose Prometheus's own metric names, these optional panels can show no data without failing mapping-pack validation.

Operational Scenarios

• When Prometheus supplies metrics for many resources, inspect scrape health, target churn, TSDB cardinality, and query pressure before trusting downstream signals.
• If a plugin's anomaly signals suddenly disappear or spike, check the Prometheus mapping pack and source telemetry together.
• MCP descriptors give agents a read-only catalog for metric-platform health.

Boundaries

• The Prometheus plugin is both a metric provider and a monitored target; it does not execute plugin-mode anomaly detectors.
• Recording rules, scrape config, and remote write settings belong to Prometheus operations and are not directly mutated by this overlay.

MCP diagnostic execution effects

The run_diagnostics tool changes diagnostic execution and cooldown state. Its descriptor declares operation_class: diagnostic_lifecycle; MCP advertises readOnlyHint=false, destructiveHint=false and idempotentHint=false. Diagnostic queries retain read-only hints. Both queries and actions continue to require mcp:invoke and the existing resource permissions/audit checks. Safety hints do not change authorization or guarantee safe retries.

MCP tool input contracts

Target, rule, and alert tools declare bounded limit/offset, substring search, and the applicable state/health/type filters. Raw PromQL tools consume end_time (and instant-query time/ts); range queries also consume start_time and step. Result limits are max_series (1–2000), max_samples (100–200000), and range-only max_data_points (200–10000). History and summary diagnostics declare lookback; static catalogs accept {}.

The advertised JSON Schema is checked before plugin dispatch. Omitted optional values retain plugin defaults. Arguments do not replace resource-instance authorization; stored connection credentials are supplied by Konduo. Open schemas are retained only for existing dynamic filters or aliases, and each advertised property describes its caller-facing meaning. Plugin result shapes can vary with product/version, available evidence, and partial failures; no stable outputSchema is promised for these dynamic responses. Client and resource coverage rules are defined in enterprise/backend/docs/plugin-contract.md in the Enterprise source repository.