case study
The Autonomous SRE Agent
A self-healing Kubernetes watchdog that runs 20+ diagnostic checks, publishes root cause analysis to Slack, and fixes issues before anyone wakes up.
the client's challenge
A production Kubernetes cluster running Sisense was a constant source of fire drills. Issues would surface at random hours — memory pressure, pod evictions, certificate expirations, disk saturation — and someone had to drop everything to triage. There was no round-the-clock visibility, no automated diagnosis, and certainly no self-healing. The team was playing whack-a-mole with a business-critical system.
our agentic approach
Rather than throwing a monitoring dashboard at the problem, Ricotek built an autonomous SRE agent — a set of Hermes cron jobs that don't just observe, but act.
Continuous Diagnostics
Python-based cron jobs running 20+ targeted checks against the Kubernetes cluster — node health, pod status, resource pressure, certificate validity, disk utilization, and application-specific metrics for Sisense.
Intelligent Triage
When a check fails, the agent doesn't just fire an alert. It correlates symptoms, identifies the root cause, and generates 2–3 proposed solutions ranked by likelihood and impact.
Self-Healing
For known failure patterns — stale pods, disk pressure, expired certificates — the agent can execute remediation automatically. It reports what it found and what it fixed. No human intervention required.
Slack-First Reporting
Every finding lands in the team's Slack channel: discovered problem, root cause analysis, proposed solutions. If the agent self-healed, it includes the remediation log. The team wakes up to answers, not alarms.
technical stack
# agent framework
Hermes cron scheduler: orchestrates check execution on configurable intervals
Python check scripts: each check is an independent, testable module
# cluster surface
Kubernetes API: node, pod, event, and resource metrics collection
kubectl + custom probes: application-level health checks for Sisense
# remediation
Automated actions: pod restarts, certificate rotation, node cordoning, disk cleanup
Guardrails: destructive actions require operator approval via Slack button
# reporting
Slack webhook: structured messages with root cause, proposed fixes, and action logs
the quantifiable result
The team went from reactive firefighting to proactive operations. The agent catches issues during off-hours, self-heals where possible, and delivers a complete diagnosis by the time anyone checks Slack in the morning. No more 3 AM pages. No more guessing what's wrong. The cluster watches itself.
Need your own SRE agent?
Every environment is different. Ricotek builds custom autonomous agents tailored to your stack, your tooling, and your team's workflow.