LLM-IN-THE-LOOP · SELF-HOSTED · FROM WATCHDOG TO AGENT

It stopped saying what to watch.
It started saying what to do.

Infra-Sentinel polls Zabbix and Prometheus with cheap, constant checks — and only when something is genuinely new does a self-hosted LLM reason about it. v1 tells an operator what to look at. v2 hands them the exact command. The road ahead: an agent that acts.

0 calls/hr
GPU stays cold until there's real news
1 alert
impact-aware, not a flood of triggers
exact fix
v2: command + where + escalation
100% local
self-hosted LLM, zero cloud egress
The middle path

Alerting stacks are usually dumb or expensive.

Forward every raw trigger and you drown the operator. Pipe everything to a cloud LLM and you pay per token to reason about noise. Infra-Sentinel splits the job: local polling does detection, a local model does reasoning — only on the delta.

DUMB

Raw-trigger firehose

Every flap, every port, every recovery becomes a page. Operators mute the channel — and miss the one that mattered.

EXPENSIVE

Everything → cloud LLM

Reasoning quality, but you pay per token to summarize noise, leak infra telemetry off-site, and depend on someone else's uptime.

INFRA-SENTINEL

Cheap detect · local reason

Constant free polling finds the delta. A self-hosted LLM reasons only on real news — then tells you exactly what to do. Nothing leaves the network.

Live pipeline

Follow a signal through the watchdog.

Two monitoring sources are diffed against persisted state; the delta runs a gauntlet of guardrails; only survivors reach the LLM and the operator. Pick a scenario and watch the path light up.

Steady state — the diff is empty, the GPU is never called.
Zabbix problems ≥ High Prometheus up == 0 diff vs baseline + dedup · cooldown silence no-op · GPU idle circuit breaker flood? threshold 8 self-hosted LLM Ollama · on-prem GPU terse alert LLM skipped Notifier → Telegram
🛰️
Infra-Sentinel
on watch
The parts worth reading

Guardrails, not glue.

The value isn't the plumbing — it's the pure, unit-tested functions that keep the watchdog useful instead of noisy, wasteful or silent.

01

Baseline seeding

The first cycle records everything already broken as the baseline and raises nothing. You're never paged for problems that predate the watchdog.

02

Dedup by objectid + cooldown

A flapping trigger emits a new event id every cycle. Sentinel keys "already alerted?" on the stable trigger id and suppresses re-alerts within DEDUP_COOLDOWN_SEC. One flapping port = one alert, not one per cycle.

03

Mass-outage circuit breaker

A flood of new problems in one cycle (MASS_OUTAGE_THRESHOLD, default 8) is the signature of a core/power failure. Sentinel short-circuits to one terse alert — no per-event LLM calls, no pager storm.

04

No fallback, ever

If the self-hosted LLM is unreachable, Sentinel does not degrade to a raw alert. It raises, aborts the cycle, and leaves state unadvanced — the same event is retried next cycle. An alert is either LLM-reasoned or it doesn't go out at all.

Two brains

From what to watchwhat to do.

v1 is in production: it turns raw triggers into one impact-aware alert. v2 is the pilot leap — a model with the infra runbook baked in, reasoning over the trigger's real metrics (not guesses), that returns the concrete fix.

v1
Observe & explain
production · Qwen 27B (base)
▲ Core switch uplink unreachable.
Distribution path for the wireless VLAN is at risk. Check the switch and its power feed before APs drop.
Turns a pile of triggers into one human-readable, impact-aware message. Great triage — but the operator still has to know the fix.
v2
Recommend the fix
pilot · sentinel-infra (runbook-baked)
▲ Gi0/2 negotiated 10 Mbps — expected 1000 (Gi class).
Cause: speed/duplex mismatch after a link flap. interface Gi0/2 speed 1000 duplex full Run from the distribution switch CLI. No recovery in 2 cycles → escalate to network on-call.
A dedicated model with the runbook in its system prompt — name↔device maps, expected link speeds by interface class, vendor CLI, escalation. Each event is enriched with the trigger's real metrics (current speed, state, counters), so it reasons on hard data, not a hunch.
The vision

The road to an autonomous infra agent.

Three rungs from a smart watchdog to an agent that works the infrastructure on its own. Two are shipped or in pilot. The third is the honest, careful part.

1

Observe & explain

Detect the delta cheaply and constantly; turn raw triggers into one impact-aware alert. Guardrails keep it quiet and safe.

✓ SHIPPED · v1
2

Recommend the fix

A runbook-grounded model reasons over real metrics and returns the exact command, where to run it, and when to escalate. Human executes.

◆ PILOT · v2
3

Act — with a human in the loop

Execute safe, idempotent remediations from a closed catalog, approved from chat, then verify the problem is gone before closing. From advisor to operator.

→ NEXT

What rung 3 still needs (the honest list)

  • Action allowlist — a closed catalog of safe, reversible ops (unlock, restart, ticket), each an idempotent, validated function.
  • Approval loop — inline approve/reject in Telegram; nothing runs unattended at first.
  • Verify return-to-BAU — re-poll after acting; if the trigger persists, escalate, don't retry-storm.
  • Audit trail — every proposed and executed action logged and reversible.
  • Statistics & trends — a daily digest of flapping/noise, historical baselines, SLO-style reporting.
  • Change-management fit — actions that respect windows, approvals and security (e.g. never auto-unlock a possibly-compromised account).
Where it honestly stands: the reasoning half is largely solved — v2 already recommends the exact fix, grounded in a runbook and live metrics. The acting half — a safe action layer with approvals and verification — is mostly unbuilt, and that's where the real engineering (and the real risk) lives. Close on the brain; deliberate on the hands.
Why the GPU stays cold

The model is a scalpel, not a firehose.

watchdog.py — the hot path
# cheap + constant: runs every cycle, no GPU
delta = diff(current, baseline)
delta = dedup_with_cooldown(delta, state)

if not delta:
    return                 # steady state → 0 GPU calls

if len(delta) >= MASS_OUTAGE_THRESHOLD:
    notify(terse_outage_alert(delta))
    return                 # flood → LLM skipped

# rare + local: only genuine, bounded news
event  = enrich(delta, zabbix_metrics)   # real numbers
alert  = llm_reason(event)              # runbook-grounded
notify(alert)

The poll loop is local and free. The LLM is only ever asked to reason about a small, deduped, enriched set of genuinely new events — and it answers with a concrete fix.

Python · stdlib-only Ollama · self-hosted LLM Zabbix + Prometheus Telegram notifier systemd · 15 min Docker Compose demo ruff + pytest CI

The trade-off, stated plainly

The scarcity that keeps the GPU cold is also the scarcity of what the system sees. The LLM never watches raw metrics or the full monitoring state — it only ever reasons over the small diff the pollers already flagged as new. If a problem doesn't surface as a change in Zabbix or Prometheus, it never reaches the model at all. Detection is the ceiling on reasoning, not the other way around.

And when the model doesn't answer, Sentinel doesn't degrade — it goes quiet. There is deliberately no fallback: if the self-hosted LLM is unreachable, the cycle raises, aborts, and leaves state unadvanced, so the same event is retried next cycle instead of shipped un-reasoned. That is the right call for correctness — no confidently wrong alert ever goes out — but it means a GPU outage and an uneventful night look identical to the operator: both are silence.

Which is also why rung 3 of the roadmap — acting, not just recommending — stays a human-in-the-loop design on purpose. A model that reasons well over a diff is not the model you hand the keys to production infrastructure, and this repo doesn't pretend otherwise.

Grounded in sentinel/analyzer.py, sentinel/cli.py and sentinel/watchdog.py — no-fallback retry semantics and diff-only snapshots, as shipped.