Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation
- 1Manipal Institute of Technology, Bengaluru
Abstract
We present Cloud Cost Intelligence, a production-ready autonomous cloud FinOps and telemetry system built around three engineering design lenses that are conventionally in tension: (1) high-frequency resource observability, (2) autonomous remediation without availability regressions, and (3) operation within a near-zero marginal monitoring cost envelope.
Traditional cloud billing monitors rely on delayed daily billing exports and static global thresholds that fail under diurnal traffic shifts, leaving idle compute instances, unattached storage volumes, and runaway serverless functions unaddressed. Our central architectural decision is a decoupled telemetry and remediation spine: multi-service AWS resource utilization metrics and billing signals are ingested asynchronously via boto3 into a single PostgreSQL (TimescaleDB) instance partitioned as compressed hypertables; anomaly scoring runs in a stateless dual-stage machine learning worker combining Holt-Winters seasonal forecasting (on a 7-day rolling window) with multivariate Isolation Forest residual scoring; and infrastructure mutations are governed by a deterministic, safety-gated remediation state machine executing 40+ automated workflows across AWS EC2, CloudWatch, IAM, and Systems Manager (SSM).
To prevent destructive false-positive terminations, adaptive per-instance dynamic thresholds (via Median Absolute Deviation) are paired with IAM tag-scoped blast-radius guards, pre-execution snapshot hooks, and real-time operator human-in-the-loop (HITL) overrides delivered via a live Dash/Plotly observability console and a low-latency WhatsApp webhook pipeline. We describe the system architecture, TimescaleDB data model, dual-stage anomaly detection mathematics, 6-state remediation lifecycle, least-privilege IAM posture, and zero-cost infrastructure envelope. Evaluated on multi-service cloud workloads within a rapid-prototyping and benchmarking cycle, the system achieves 94.2% anomaly detection accuracy, reduces manual FinOps oversight by 90%+, lowers net cloud compute overhead by 15%, and uncovers $500+/month in recurring infrastructure waste.
Related
- Solar photovoltaic module multi-class anomaly detection using MobileNet architecture with skip connections
- A blockchain-enhanced CNN architecture for robust anomaly detection in industrial and IoT cyber-physical systems
- Pilot Study Protocol — The Athena Principle: Error Detection, Reflective Pause, and Compassion-Centred Remediation in a Reciprocal AI Architecture (Draft 0.4)
- Your AI Agent Is Now an IAM Principal: Identity Architecture for Autonomous Agents
- Lightweight anomaly detection via knowledge distillation from ensemble teachers
BibTeX
@misc{goel2026cloud,
title = {{Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation}},
author = {Shaurya Goel and Samyukta Sreekanth and Poojitha Madan},
howpublished = {Zenodo},
year = {2026},
month = oct,
doi = {10.5281/zenodo.23622226},
url = {https://zenodo.org/records/23622226},
note = {CC BY 4.0},
}
RIS
TY - UNPB
TI - Cloud Cost Intelligence: An Autonomous FinOps Architecture with Dual-Stage Time-Series Anomaly Detection and Safety-Gated Cloud Remediation
AU - Shaurya Goel
AU - Samyukta Sreekanth
AU - Poojitha Madan
PY - 2026
DA - 2026/10/24
PB - Zenodo
DO - 10.5281/zenodo.23622226
UR - https://zenodo.org/records/23622226
AB - We present Cloud Cost Intelligence, a production-ready autonomous cloud FinOps and telemetry system built around three engineering design lenses that are conventionally in tension: (1) high-frequency resource observability, (2) autonomous remediation without availability regressions, and (3) operation within a near-zero marginal monitoring cost envelope. Traditional cloud billing monitors rely on delayed daily billing exports and static global thresholds that fail under diurnal traffic shifts, leaving idle compute instances, unattached storage volumes, and runaway serverless functions unaddressed. Our central architectural decision is a decoupled telemetry and remediation spine: multi-service AWS resource utilization metrics and billing signals are ingested asynchronously via boto3 into a single PostgreSQL (TimescaleDB) instance partitioned as compressed hypertables; anomaly scoring runs in a stateless dual-stage machine learning worker combining Holt-Winters seasonal forecasting (on a 7-day rolling window) with multivariate Isolation Forest residual scoring; and infrastructure mutations are governed by a deterministic, safety-gated remediation state machine executing 40+ automated workflows across AWS EC2, CloudWatch, IAM, and Systems Manager (SSM). To prevent destructive false-positive terminations, adaptive per-instance dynamic thresholds (via Median Absolute Deviation) are paired with IAM tag-scoped blast-radius guards, pre-execution snapshot hooks, and real-time operator human-in-the-loop (HITL) overrides delivered via a live Dash/Plotly observability console and a low-latency WhatsApp webhook pipeline. We describe the system architecture, TimescaleDB data model, dual-stage anomaly detection mathematics, 6-state remediation lifecycle, least-privilege IAM posture, and zero-cost infrastructure envelope. Evaluated on multi-service cloud workloads within a rapid-prototyping and benchmarking cycle, the system achieves 94.2% anomaly detection accuracy, reduces manual FinOps oversight by 90%+, lowers net cloud compute overhead by 15%, and uncovers $500+/month in recurring infrastructure waste.
ER -