Skip to content

Kameswara Sai Srikar MandaM.S. Data Science, Northeastern · graduating Dec 2026

I ship ML and LLM systems to real users, and measure where they break.

I'm Srikar, an M.S. Data Science student and teaching assistant at Northeastern. At Launch Plan, until July 2026, I built for the people using a K-12 financial-literacy app through its pilot: a dashboard for school counselors, a college-fair kiosk for students, and LangGraph agents and a RAG search for the teaching team. Before that I led full-stack work on an AI-tool marketplace used by more than 10,000 people a month.

Open to forward-deployed engineer, data scientist and software engineer roles.

Bar chart of how often three open models repeat a planted false claim or obey a planted instruction once it is retrieved.
One planted passage in 10,000 flipped 21–42% of the answers it targeted.Read the case study →

What the studies found

Each study asks one question, measures it on real data or real models, and keeps the results that came out wrong.

Measured, not argued

Most claims about how AI systems break are a screenshot of one bad answer. These put a number on one failure at a time, with the same rules each time: one factor at a time, a noise floor from a repeat run, and predictions written down before the full run.

The series: Measuring LLM Systems

How often does checking early find a winner that isn't there?

Both versions of the test are identical, so every winner is a false one. It should be 5%.

Standard test, checked each time18.7%
A test built for peeking (mSPRT)0.23%

20,000 simulated tests at real headline-test sizes · how it was measured

Selected work

All projects

One Bad Passage in 10,000

A retrieval-poisoning study: plant one false or malicious passage among 10,000 that a question-answering system reads from, and measure who it reaches, how often three open models repeat it, whether it spreads, and what simple defences cost in accuracy.

  • Python
  • RAG
  • LLM security
  • Ollama
  • sentence-transformers

The First Call Is Easy

How reliably do local LLMs use tools? Three models made the right first call about 80% of the time but finished only 36–69% of tasks: they stopped after one step of a chain and reported errors they could fix. More tools, vague names and loose schemas barely mattered.

  • Python
  • Tool calling
  • Ollama
  • Agents
  • LLM evals

Prompt Brittleness

How often does a harmless rewording change an LLM's answer? 300 maths, multiple-choice and extraction items on three local models, plus a 30B thinking model: 5–8% of answers move per rephrasing, shuffled options move the most, and the best-looking prompt doesn't hold up on new items.

  • Python
  • Prompt evals
  • Ollama
  • GSM8K
  • MMLU
  • LLM evals

whygit

Looks at a Git repository and says in plain English what state it's in and how to fix it: merge conflicts, a rebase stopped halfway, a rejected push, a 100 MB CSV about to be committed. A command-line tool and an MCP server for coding agents, on PyPI. It never changes the repository, and the tests prove it.

  • Python
  • Git
  • MCP
  • CLI
  • GitHub Actions

bostonmove

Getting around Boston from live public data: MBTA departures and service alerts, Bluebikes bike and dock counts, and door-to-door options by foot, bike or train, including one change between subway lines. A command-line tool and an MCP server, so coding agents answer with real numbers instead of guessing. On PyPI and the MCP registry.

  • Python
  • MCP
  • CLI
  • MBTA API
  • GitHub Actions

Keystone

Shows what a research paper is really standing on: the work it builds on, the work it argues with, and the assumptions it never defends. It reads all three straight from the paper's LaTeX, with no language model involved, and pins each one to the exact spot on the page. It covers 234 papers so far and publishes its own error rate: 80% precision on SciCite, 69% on ACL-ARC.

  • Python
  • LaTeX parsing
  • NLP
  • Next.js
  • pdf.js
  • Redis

Experiment Forensics

Re-analyses 22,743 real headline A/B tests from Upworthy and a 64,000-customer email test: how small real effects are, what checking early costs, how much winners shrink, and when variance reduction helps. The plan was locked before the confirmation data was opened, and every check replicated.

  • Python
  • Statistics
  • A/B Testing
  • Empirical Bayes
  • statsmodels

Duskline

A racing game that runs in your browser, with nothing to install. Everything is made in code: the skies are shaders, the cars are modelled from cross-sections, and the engine, tyres, wind and music are synthesised live. 12 worlds, 14 cars and 4 modes, all built around drifting to charge your nitro. Works on phones, and offline.

  • TypeScript
  • Three.js
  • WebGL
  • Web Audio API
  • Next.js

GitGotchi

A companion that lives in your git repository and grows with how you commit. Commits written by AI agents slowly possess it, and every form is a 3D model drawn as pixel art in the terminal. On PyPI, with 200+ downloads.

  • Python
  • NumPy
  • Git Hooks
  • AI Agents
  • CLI Tool

Experience

Full history
  1. September 2026 - Present

    Teaching Assistant, DS 3500: Advanced Programming with Data · Northeastern University

    • Teaching assistant for DS 3500, Advanced Programming with Data, working with Professor Deahan Yu.
    • Helping a large class through Git for the first time, from installing and authenticating to their first merge conflicts.
  2. January 2026 - July 2026

    Data Scientist · Launch Plan

    • Worked on a K-12 financial-literacy platform: a mobile app that went into a TestFlight pilot, plus a dashboard for school counselors.
    • Built the counselor dashboard from scratch: cohort analytics behind a privacy floor, and FERPA-gated messaging to students over push and email.
    • Built Reality Check, a kiosk app for college fairs that shows students what a career and college choice really costs.
    • Led a security and correctness sweep before the pilot that closed 25 audit findings, 2 of them critical.
    • Built LangGraph workflows and MCP server integrations that let LLM agents write and tailor lessons using internal tools and outside APIs, taking content formatting and assessment assembly off the teaching team.
    • Replaced a keyword search educators didn't trust with a RAG pipeline on Pinecone and custom embedding models.
    • Built a data layer with FastAPI, PostgreSQL and Airflow that pulls in content from several school districts.
    • About 130 commits across the mobile app, API and data layer.
    Read the case study →
  3. January 2026

    Quantum Winter School (Mentee) · Massachusetts Institute of Technology

    • On-site program in Cambridge, MA, working hands-on with quantum computing in Qiskit.

Latest writing

All writing

On GitHub

Get in touch

Questions about my work, a role, or a project idea. Send a message here or email me directly.

Srikarmks@gmail.com