Skip to main content

Projects

ConsoleTrack

ConsoleTrack catches out-of-band AWS changes, records who made them, how, and which Terraform-managed resource they touched. It links later EKS incidents back to the change that most likely caused them. A read-only AI agent triages incidents through two MCP servers. Every AWS-side fix requires human approval.

Try the demo · Watch the 3-minute walkthrough · View the code

How it works

  1. Intake: CloudTrail → EventBridge → SQS catches console, CLI, CloudShell, and SDK writes, with a canary heartbeat to prove the pipeline is alive.
  2. Change records: each change is matched to its Terraform address and analyzed, then a reviewer can Accept it with a note, Revert it, or Codify it.
  3. Incidents: Alertmanager opens an incident, and the triage agent investigates through a read-only k8s-mcp and aws-mcp, then proposes a fix.
  4. Remediation: a policy gate decides. One narrow cluster fix, raising a memory limit after an OOM kill, runs automatically behind a deterministic validator, a server-side dry run, rollout verification, and automatic rollback. Everything else waits for a human.
  5. Observability: one OpenTelemetry trace follows each incident from alert to rollout, in Grafana (Tempo, Loki, Prometheus).

Design principles

  • The agent only reads. Both MCP servers are read-only by design: an allowlist of API calls, checked twice; IAM scoped by path; RBAC verified on a real cluster.
  • Untrusted text stays untrusted. Text a user could have written, such as a resource tag or CloudTrail parameter, is returned to the agent labeled as untrusted data, a defense against prompt injection.
  • Autonomy is earned, not assumed. The OOM fix graduated to automatic only after an eval: across 20 runs, the agent proposed the correct fix every time. The eval also showed that model confidence was poorly calibrated, so safety lives in the deterministic layers, not in the model's self-assessment.

Built with

Terraform · Amazon EKS · Pod Identity · Helm and Helmfile · FastAPI · PostgreSQL · React · Claude API · Model Context Protocol · OpenTelemetry · Grafana · GitHub Actions with OIDC

Built in 13 days as the final project for the DataTalks.Club AI Dev Tools Zoomcamp.

Read more


FaultTrack

FaultTrack looks at the metrics around an alert and gives on-call a first guess before anyone opens a dashboard: the fault type (CPU, memory, disk, socket, network delay or packet loss) with class probabilities, and the service it most likely started in. The fault type comes from a trained classifier. The suspect service comes from a simple rule. Each answer is labeled with where it came from and its own accuracy.

Try the live API · Results page · View the code

How it works

  1. Data: RCAEval RE2, 270 fault-injection cases across three microservice systems (Online Boutique, Sock Shop, Train Ticket), metrics only.
  2. Features: how far each metric moved in the two minutes after the alert, compared with the five minutes before it, scaled with a floor for near-constant metrics and log-compressed, then grouped by metric kind so one model works across systems with 11 or 68 services.
  3. Model: a random forest, chosen over logistic regression, a decision tree and XGBoost on the same grouped folds.
  4. Serving: a FastAPI service on Fly.io. Send the metrics around an alert as a CSV, with the detection time, to POST /predict/case, and get both answers back, labeled.

Results

  • Macro F1 0.80 on the systems it was trained on (grouped 5-fold cross-validation).
  • About 0.57 on a system it has never seen (leave-one-system-out), and 0.41 on the system least like the others.
  • The root-cause rule names the right service in 85–97% of alerts per system, with no training.

Design principles

  • Evaluation can't leak. Each case contributes five windows, so every split is grouped by case, including the inner hyperparameter search. A Claude Code hook stops the coding agents from writing row-level splits, and tests check the grouping.
  • The headline doesn't hide the gap. The unseen-system score sits next to the headline, with the reason it's lower.
  • Model and rule are labeled separately. The API never presents a rule's answer as a model's.

Built with

Python · pandas · scikit-learn · FastAPI · Docker · Fly.io · GitHub Actions

Built in under two weeks as the midterm project for the DataTalks.Club ML Zoomcamp. Dataset: RCAEval (Pham et al., WWW 2025), doi:10.5281/zenodo.14590730, CC BY 4.0.

Read more

A write-up on what FaultTrack taught me about evaluation is coming soon.


PathTrack (in progress)

PathTrack predicts a speed test's final throughput from its first second or two of measurements, so a test could stop early without losing accuracy. It's trained on real Internet speed tests from M-Lab, following the approach of TurboTest (NSDI '26), on an AWS ML pipeline: SageMaker for training and the model registry, Kubernetes for serving.

Coming early 2027, as the capstone project for the DataTalks.Club ML Zoomcamp.