Gateways, monitoring, serving, release gates and fine-tuning checks. 6 projects, each with its code on GitHub.
One entry point for every LLM call: model routing by difficulty, per-tenant budgets and rate limits, fallback chains, caching, and guardrails that redact secrets before they leave.
Monitoring built for LLM applications: cost per feature and per success, latency percentiles that exclude cache hits, grounding rates, and prompt drift detected by PSI without ever storing a prompt.
Multi-model serving: A/B testing with sticky assignment, canary rollout, shadow traffic, per-version SLOs and auto-rollback that knows the difference between a bad canary and an upstream outage.
The four things that break a deployed model: point-in-time feature leakage, training/serving skew, undocumented models, and unlawful data reuse. A release gate that refuses rather than warns.
AIOps: Drain log-template extraction, anomaly detection robust to the outliers it is looking for, and correlation that turns forty alarms into one incident with a suspect.
The parts of fine-tuning that go wrong before the GPU is touched: VRAM budgeting, loss masking that stops the model learning to generate prompts, leak-free splits, schedules, and before/after evaluation.