Apache Flink Plugin

The Apache Flink plugin registers a Flink cluster as a Konduo resource. It uses the JobManager REST API and a linked Prometheus metric source to expose job execution, checkpoints, throughput, backpressure, and JVM state.

Highlights

• Reads current cluster and active-job state from the JobManager REST API.
• Provides 19 Flink logical metrics through a Prometheus mapping pack.
• Uses 16 default panels for cluster capacity, throughput, checkpoints, and JVM resources.
• Separates current state from bounded historical metric evidence.
• Provides managed alert rules for backpressure, task slots, checkpoints, job restarts, and JVM pressure.

Before Registration

1. Prepare a JobManager REST URL that is reachable from the Konduo backend. The default is http://localhost:8081, but in a container deployment localhost can mean the Konduo backend container itself.
2. If the REST API requires Bearer authentication, prepare an API token. The token is stored as sensitive configuration and is used only in the Authorization header of diagnostic requests.
3. Enable the Prometheus reporter on the JobManager and every TaskManager, then confirm that Prometheus scrapes those endpoints.
4. Link the target Prometheus resource to the Flink resource as its metric source.
5. If metrics.scope.* or the reporter prefix is customized, prepare mapping rules for that environment instead of relying on the default pack.

Connection Settings

Field	Required	Default	Description
endpoint	yes	http://localhost:8081	Base URL of the JobManager REST API
api_token	no	none	Bearer token sent with REST API requests
Metric-source link	for metrics	none	prometheus-plugin resource that executes time-series queries

A URL that works in a browser can still be unreachable from the Konduo backend. For container deployments, verify service DNS, published ports, and firewall rules from the backend network.

Registration And Validation

1. Enter the endpoint and, when required, api_token, then run Connection Test.
2. Save the resource and confirm the Flink version, registered TaskManagers, and total/available task slots in cluster diagnostics.
3. In Prometheus, confirm that the JobManager and every TaskManager target is UP.
4. Link that Prometheus resource to Flink as its metric source.
5. Even with no running job, confirm that cluster and JVM panels contain current samples.
6. With a test job, validate job/task labels on input/output rate, busy time, backpressure, and checkpoint panels.
7. Confirm that nine managed alerts are registered. Enable checkpoint-age and checkpoint-duration rules only after defining workload-specific baselines.

REST connectivity and Prometheus collection validate separate paths. A healthy REST diagnosis does not prevent empty dashboards or time-window diagnostics when reporters or metric linkage are misconfigured.

Dashboard And Metrics

The default dashboard has four areas:

• Cluster overview: registered TaskManagers, running jobs, total and available task slots, and slot utilization
• Workload and backpressure: input/output records per second, task busy time, and backpressure
• Checkpoints and recovery: checkpoint duration, age, failures, and job restarts
• JVM resources: JobManager and TaskManager CPU and heap utilization

Job, task, and checkpoint panels can legitimately have no data when no job is running. If cluster overview and JVM panels still have data, do not classify that state as a complete metric collection failure.

The default mapping assumes the Flink Prometheus reporter's flink_ prefix and default logical scopes. If all panels or one role-specific group is empty, check the Prometheus target, actual metric names, labels, and active mapping pack in that order.

Diagnostics

The diagnostic summary combines current REST evidence with metric-evidence readiness. Detail views are split into:

• Cluster: JobManager REST reachability, registered TaskManagers, and task-slot capacity
• Jobs: active job state and failure or restart signals
• Checkpoints: the latest checkpoint outcome for active jobs
• Performance evidence: backpressure, throughput, checkpoint, and JVM logical metrics

Current diagnosis reads /overview, /jobs/overview, and /jobs/{jobid}/checkpoints within bounded timeout and response-size limits. The current REST diagnostic budget is eight seconds, each response is limited to 2 MiB, and checkpoint inspection is limited to ten active jobs.

A FAILED job is historical context and does not by itself raise a current warning. Active states such as FAILING, RESTARTING, and SUSPENDED affect the live stability result. Batch and stateless jobs may not use checkpointing, so missing checkpoint activity is not automatically a failure.

Time-window diagnosis delegates logical metric queries to the linked Prometheus resource. When the metric source is absent or cannot serve the query, the plugin reports unavailable evidence and a follow-up route instead of assuming a healthy state.

The diagnostic run action refreshes current evidence only. The plugin does not provide persistent storage for previous diagnostic results.

Alert Rules

flink-alert-rules-v1 provides nine rules:

Rule	Severity	Default condition	Pending duration	Default state
Task backpressure high	warning	above 50% over 5 minutes	3 minutes	enabled
Task-slot utilization high	warning	at least 90% over 10 minutes	5 minutes	enabled
Checkpoint failures detected	critical	at least one increase over 5 minutes	1 minute	enabled
Job restarts detected	warning	at least one increase over 10 minutes	2 minutes	enabled
Latest checkpoint age high	warning	at least 900 seconds over 10 minutes	5 minutes	disabled
Checkpoint duration high	warning	at least 60 seconds over 10 minutes	5 minutes	disabled
TaskManager JVM heap high	warning	at least 85% over 5 minutes	3 minutes	enabled
JobManager JVM heap high	warning	at least 85% over 5 minutes	3 minutes	enabled
JobManager JVM CPU high	warning	at least 90% over 10 minutes	5 minutes	enabled

Checkpoint age and duration vary significantly by job interval and state size, so those two rules are disabled by default. Before enabling them, tune the thresholds using the target job's checkpoint interval, state backend, and normal duration.

An alert is an investigation starting point, not a standalone conclusion. For example, correlate a backpressure alert with downstream throughput, busy time, checkpoint duration, and TaskManager capacity.

Example Workflows

Repeated job restarts

1. Inspect the current job state and recent exception.
2. Check recent checkpoint failures and the last successful checkpoint.
3. Verify checkpoint storage latency and reachability.
4. Compare TaskManager CPU, heap, and backpressure over the same interval.
5. Adjust restart strategy or parallelism only after confirming the evidence.

Throughput degradation

1. Compare input and output record rates.
2. Locate operators with high backpressure and tasks with high busy time.
3. Check task-slot headroom and resource skew between TaskManagers.
4. Inspect sink or external dependency latency before deciding to scale.

Slow or failing checkpoints

1. Identify the affected job and its latest successful and failed checkpoints.
2. Read checkpoint duration, age, and failure increase together.
3. Compare state size, alignment delay, backpressure, and storage health.
4. Do not hide the root cause with repeated restarts.

Management Boundary

• The CE plugin observes and diagnoses Flink. It does not automatically deploy, cancel, savepoint, or rescale jobs.
• Missing metrics do not prove that Flink is down. Evaluate REST state and collection state separately.
• Never expose the API token in diagnostic results, logs, or UI descriptions.
• Default alert thresholds are starting points and must be reviewed for the workload.

Troubleshooting

Symptom	Check in this order
Only REST diagnosis fails	Check endpoint, JobManager state, backend network path, and firewall
REST returns 401 or 403	Check the Bearer token value and expiry, proxy authentication forwarding, and permission for the target REST route
REST diagnosis times out	Check backend-to-JobManager latency, proxy behavior, and JobManager load; the diagnostic budget is eight seconds
Response format is invalid	Check whether a proxy error page replaced JSON, REST API compatibility, and whether the response exceeds the 2 MiB limit
Every metric panel is empty	Check Prometheus targets, JobManager/TaskManager reporters, and the metric-source link
Only job/checkpoint panels are empty	Check for active jobs and checkpoint usage; this can be normal on an idle cluster
Only one role or job is empty	Check actual metrics.scope.*, the flink_ prefix, labels, and mapping-pack rules
Time-window diagnosis is unavailable	Verify the Prometheus link and range-query support for the required logical metric

---

Apache Flink Enterprise Overlay

The Apache Flink Enterprise overlay adds multi-signal anomaly rules and read-only MCP descriptors to the Community Flink resource plugin. Community continues to own connectivity, JobManager REST diagnostics, dashboards, logical metrics, mapping packs, and managed alerts.

Before Registration

1. Configure JobManager REST connectivity and optional Bearer authentication as described in the Community manual.
2. Link the Prometheus resource that collects the JobManager and every TaskManager to the Flink resource.
3. Confirm that the Enterprise license enables mcp.gateway and anomaly.engine.
4. Grant users who read anomaly rules at least the viewer role and flink-plugin.anomaly.read.
5. Give MCP callers both an allowed MCP read scope and access to the target Flink resource instance.

MCP and anomaly analysis do not replace the Flink REST connection. Validate REST state, Prometheus collection, and logical-metric mapping separately.

Anomaly Rules

flink-anomaly-rules-v1 provides five rules. Every rule uses all condition mode, so all listed conditions must match in the same evaluation context.

Rule key	Severity and score	Conditions	First checks
flink.worker_availability_risk	critical, 0.98	running jobs at least 1; registered TaskManagers below 1	TaskManager registration, heartbeat, network, JobManager leadership, and task slots
flink.checkpoint_restart_cascade	critical, 0.94	checkpoint-failure increase above 0; job-restart increase above 0	job exception, latest completed/failed checkpoint, and checkpoint storage
flink.backpressure_capacity_saturation	warning, 0.86	backpressure at least 50%; task-slot utilization at least 90%	constrained operator, upstream/downstream throughput, slots, and parallelism
flink.taskmanager_resource_pressure	warning, 0.82	TaskManager CPU at least 90%; heap utilization at least 85%	per-TaskManager skew, GC, busy time, and backpressure
flink.throughput_stall_pattern	warning, 0.80	input at least 1 record/s; output at most 0.001 record/s; busy time at least 85%	sink, asynchronous I/O, operator exceptions, and per-task skew

These rules do not replace Community managed alerts. Managed alerts evaluate individual thresholds and pending durations, while Enterprise rules combine logical metrics to provide incident-triage evidence.

Job and checkpoint metrics can be absent on an idle cluster. Do not interpret a missing series or unresolved mapping as healthy or zero; inspect metric linkage and mapping state first.

MCP Resources And Tools

The MCP catalog provides eight read resources and ten read tools.

Kind	Surfaces
Resources	monitoring overview, current diagnostics, historical diagnostics, metric catalog, mapping-pack catalog, managed alerts, anomaly rules, and software inventory
Diagnostic tools	monitoring overview plus summary, cluster, job, checkpoint, and performance diagnostics
Metric tools	logical-metric query resolution and mapping-pack resolution
Rule tools	managed-alert and Enterprise anomaly-rule metadata

The metrics_query_resolve tool accepts:

Input	Required	Value
logical_metric_key	yes	A logical key from the Community Flink metric catalog
query_mode	no	instant or range

MCP descriptors expose query routes only. The diagnostics/summary/run refresh is a separately RBAC-protected write action and is deliberately excluded from the read-only catalog. Job deployment, cancellation, savepoints, and rescaling are also not exposed through MCP.

Operational Workflows

Worker availability risk

1. Confirm the running-job and registered-TaskManager counts.
2. Read TaskManager registration and task-slot evidence from JobManager REST diagnostics.
3. Check TaskManager process, heartbeat, network, and resource state.
4. After workers recover, verify available slots and job recovery before deciding whether to restart.

Checkpoint and restart cascade

1. Identify the affected job and its recent exception.
2. Compare completed and failed checkpoints, duration, and storage reachability.
3. Read backpressure, TaskManager CPU/heap, and restart increase over the same interval.
4. Do not repeatedly change restart strategy or parallelism before confirming the cause.

Throughput stall

1. Compare input, output, and busy time by job and task over the same interval.
2. Inspect the stalled operator chain, sink, asynchronous I/O, and external dependencies.
3. Compare backpressure and slot headroom to distinguish skew from overall capacity shortage.

Troubleshooting

Symptom	Check in this order
EE extension or MCP entries are absent	Check Enterprise license, mcp.gateway, anomaly.engine, plugin version, and contribution loading
Anomaly-rule query is denied	Check target-resource access, the viewer role, and flink-plugin.anomaly.read
No anomaly rule can be evaluated	Check the Prometheus link, Flink reporter targets, logical-metric mappings, and range-query support
Only job-related rules are unevaluated	Check for running jobs, checkpoint use, job/task labels, and default scopes
MCP metric resolution fails	Confirm that logical_metric_key exists in the current catalog and use instant or range for query_mode
Diagnostic refresh is absent from MCP	This is the intended read-only boundary; use Konduo's RBAC-protected diagnostic action

Edition Boundary

The anomaly pack, MCP descriptors, and Enterprise localization remain in the EE overlay. Flink mutations and diagnostic execution do not bypass the read-only MCP surface; they remain behind existing Konduo API, RBAC, and audit boundaries.