Skip to content
M/S
Resume

MADHAV M S
AI Infrastructure Engineer · CS @ CIT · Data Science @ IIT Madras

I build the layerunder the model.

Capacity governance for LLM inference, retrieval that cites its sources, vector search from the paper up, and memory for coding agents. I measure everything I build and publish the numbers, including the ones that didn't hold up.

01About

Systemsfirst, thenintelligence.

→I'm a Computer Science student at Chennai Institute of Technology, and I'm doing a BS in Data Science at IIT Madras at the same time.

I care about the parts of AI systems that break in production. That means how requests get admitted when the GPUs are full, why an embedding service runs out of memory, whether a retrieved passage really supports the answer, and what an agent should remember between sessions.

I learn these things by building them and measuring them, and I try to be honest about what I find — writing up what worked, what didn't, and where an idea looked better on paper than it held up in practice.

Before this I did two internships at Invisibl Cloud Solutions, building production RAG pipelines, multi-agent workflows and a self-healing data pipeline.

02Projects

Six thingsI built andmeasured.

Each one ships with a README that reports the numbers, including where they fell short.
01 / 06Shipped

SluiceQoS-Aware Capacity Governance for Self-Hosted LLMs

Stops one noisy tenant from eating a paying customer's SLA.

Sluice is a FastAPI admission-control proxy for multi-tenant vLLM clusters. For every request it decides whether to pass, degrade, queue, fall back to a smaller model or reject. It combines GPU cache pressure, queue depth and per-tier SLA-violation rates to make that call. Redis coordinates fallback routing and cross-replica queueing, a PostgreSQL decision ledger records why each call was made, and Prometheus handles telemetry. It also has an offline replay tool that tests policy changes against recorded traffic.

Headline metric

Guaranteed-tier SLA violations cut from 30.0% to 2.0% under sustained overload, at ~0.006 ms median proxy overhead.

Synthetic benchmark against a mock backend.

  • Python
  • FastAPI
  • vLLM
  • Redis
  • PostgreSQL
  • Prometheus
  • Docker
  • Kubernetes
02 / 06Shipped

DocuQueryAsk Your Documents, Get Cited Answers

Production RAG that shows the exact page and passage behind every answer.

DocuQuery is a full-stack RAG platform. It has an async FastAPI backend, a PostgreSQL + pgvector (HNSW) store, and SSE-streamed answers from five LLM providers with document-, page- and chunk-level citations. Authenticated routes keep each user's documents isolated from everyone else's. When production hit an out-of-memory crash on Railway, I replaced the embedding stack with ONNX Runtime INT8 (bge-small-en-v1.5), then benchmarked batch sizes from 1 to 64 to choose the default.

Headline metric

Retrieval latency P50 11.8 ms / P95 14.1 ms · batched embeddings run 6.17× faster than unbatched · 79/79 backend tests passing.

  • Next.js
  • TypeScript
  • FastAPI
  • PostgreSQL + pgvector
  • ONNX Runtime
  • SSE
  • Docker
03 / 06In development

CairnGit-Native Memory for AI Coding Agents

Leave a marker so the next agent doesn't start from zero.

Cairn turns what a coding agent learns in a session (a failed approach, a build-system quirk) into reviewed plain-Markdown entries committed to the repo. It runs a capture → reflect → curate → inject pipeline over a schema-validated Markdown/YAML store, and its adapters work with both Claude Code and opencode. The design is adapted from ACE (Agentic Context Engineering).

Headline metric

A CLI (init, validate, context, review) with deterministic duplicate- and contradiction-detection gates, plus an eval harness that scores extraction against gold-standard fixtures in CI.

  • Python
  • Anthropic SDK
  • Pydantic
  • JSON Schema
  • Typer
  • Agent Tooling
04 / 06Shipped

VigilarchTamper-Evident Offline Incident Ledger

Proves when a record was written without a server or a trusted clock.

Vigilarch is an append-only ledger for incident reports at disconnected sites such as mines, tunnels and ports. When two nodes meet, they co-sign each other's chain head. That bounds each record's creation time between attestations, so any backdating shows up as a detectable fork that can be traced to a key. It's written as a Rust workspace of nine crates, and the core also compiles to WebAssembly for the live browser demo.

Headline metric

3 scenarios × 1,000 seeds, 3000/3000 passing · 158 native tests.

  • Rust
  • WebAssembly
  • Cryptographic Attestation
  • Offline-First
  • Simulation Testing
05 / 06Shipped

LoamA Readable, Instrumented HNSW Vector Index

The index behind every RAG pipeline, built line by line from the paper.

Loam is HNSW vector search in pure Python and NumPy. Every benchmark is checked against a brute-force index, and a search trace shows where the ef budget goes, one hop at a time. It includes ablations on neighbor selection and on whether the hierarchy actually helps.

Headline metric

recall@10 of 0.997 at ef=64 on GloVe-25 (10k vectors), using ~10% of the distance computations of an exact scan.

  • Python
  • NumPy
  • ANN Search
  • Benchmarking
06 / 06Shipped

MarginalisIs "Run It When the Grid Is Green" Right?

Average carbon intensity is the wrong signal for scheduling load.

Marginalis estimates marginal and average CO₂ emissions factors from EIA-930 hourly grid data for three US balancing authorities (2019–2026), using first-difference regression. The method and hold-out split were frozen before any evaluation. The effect held in MISO but not in ERCOT or CAISO, and the README reports all three.

Headline metric

MISO: scheduling by marginal cost avoided 230 kg CO₂/MWh (95% CI 44–420) on 2025 hold-out data, and the result held again in 2026.

  • Python
  • FastAPI
  • PostgreSQL
  • React
  • TypeScript
  • Econometrics

03Experience & Education

WhereI'veshipped.

Apr – Jun 2026

AI Agent Engineer Intern

Invisibl Cloud Solutions Pvt Ltd

  • Built multi-agent RAG workflows with LangGraph, MCP and FastAPI for protocol retrieval, validation and root-cause troubleshooting, using ChromaDB and Sentence Transformers for semantic retrieval.
  • Developed an autonomous hardware-fleet diagnosis workflow that analyzed device telemetry and syslogs to find root causes. It enforced a deterministic 85°C thermal guardrail that blocked unsafe reboot actions.
  • Engineered a self-healing transaction-data pipeline (LangGraph + Pydantic) that detected and corrected type and arithmetic errors before they reached downstream reporting. In one case it fixed an incorrect total from 900 to 1,000 within a 3-cycle validation loop.

Nov – Dec 2025

Platform Engineer Intern

Invisibl Cloud Solutions Pvt Ltd

  • Built a natural-language analytics tool that lets non-technical users query CSV/XLSX data in plain English and get charted answers within seconds. It was evaluated across 50+ files holding millions of records.
  • Engineered a multi-format RAG pipeline over 50 documents (~3,000 pages) using Sentence Transformers, semantic chunking and ChromaDB top-5 retrieval, with retrieval evaluations and input/output guardrails.
  • Developed an AWS Strands agentic RAG workflow with 5 tools and evaluated it on 1,000 test cases, reaching a 95% success rate before deployment.

Jun 2026 – Present

Member

Artificial Intelligence Centre of Excellence, Chennai Institute of Technology

  • Contribute to applied AI research, workshops and project incubation run by the institute's AI CoE.
  • Collaborate with peers on applied AI/ML projects.

Education

  • BE Computer Science & Engineering

    Chennai Institute of Technology · 2024 – 2028 · CGPA 8.4

  • BS Data Science & Applications

    IIT Madras · 2024 – Present

04Skills

Thetoolbox.

AI Infrastructure

  • LLM serving & admission control
  • vLLM
  • capacity/QoS policies
  • LLM evaluation
  • agent memory & context engineering
  • AI safety guardrails

AI Engineering

  • RAG
  • multi-agent systems
  • LangGraph
  • LangChain
  • AWS Strands
  • MCP
  • embeddings
  • semantic search
  • ONNX Runtime

Retrieval & Data

  • pgvector
  • ChromaDB
  • HNSW
  • semantic chunking
  • metadata filtering
  • ETL pipelines
  • Pandas
  • NumPy
  • statistics & regression

Backend

  • FastAPI
  • Flask
  • REST APIs
  • SSE streaming
  • async Python
  • auth & tenant isolation
  • PostgreSQL
  • Redis

Languages

  • Python
  • TypeScript/JavaScript
  • SQL
  • C/C++
  • Rust

Infra & Ops

  • Docker
  • Kubernetes
  • Prometheus
  • AWS (EC2, S3)
  • CI/CD
  • GitHub Actions
  • Linux
  • Vercel
  • Railway
  • Render

05Achievements & Community

Off theclock.

20+ public GitHub repositories spanning AI/ML tooling and backend infrastructure

Clubs

Member, Quant Club, Chennai Institute of Technology · Feb 2026 – Present. Analyze market data for statistical trading-strategy design.

Hackathons

Smart India Hackathon 2026·The Great Agent Hackathon·AI Builders Hackathon·Razorpay AI Buildathon·IBM Z Datathon

06Contact

Let's talkshop.

I like talking about inference, retrieval, evals and agent tooling. If you're working on the layer under the model, I'd like to hear about it.