Static taint-analysis toolkit for finding injection-to-impact bugs in agent frameworks and MCP servers (JavaScript/TypeScript and Python). It traces tool arguments — attacker-controllable, including via prompt injection of the calling agent — into dangerous sinks (shell, argv, filesystem paths, outbound URLs, eval/deserialization, SQL), then ranks and helps you triage the results for coordinated disclosure.
Built on Semgrep OSS, plus the glue it lacks: a cross-function reach walk, chunked re-scans to beat Semgrep's silent fixpoint timeout, and a priority queue tuned to the shapes real agent tool bugs take.
| Experiment 1: MCP servers | Experiment 2: agent frameworks | |
|---|---|---|
| Targets | Top 200 npm packages by weekly downloads, filtered to 141 servers that register tools | 100 frameworks curated from GitHub topic searches, scanned least-used first |
| Scanned | 141 of 141 | 99 of 100 (agent-native skipped for size, mastra run alone) |
| New findings | 1: crewly, published as GHSA-v7q6-5mpg-q47w (High, 8.8; CVE pending) | 3: validated and submitted to maintainers on 2026-11-04 (CVEs pending) |
| Results | SUMMARY.md |
METHODOLOGY.md, Part 2 and exp2-frameworks/ |
The three experiment 2 findings are under embargo until a fix ships or 2027-02-02, whichever comes first. Keep their details out of anything public until then.
Experiment 1 only recognised MCP tool registrations. Commit e528437 extended the same pipeline:
- Python taint sources:
@tool,@function_tool,@kernel_function,.tool_plain,.agent_tool, and the parameters of_run,_arunandforward. - JS/TS taint sources: LangChain.js
tool(fn, {schema}),DynamicTool,DynamicStructuredTooland$F.from_function(...). Vercel AI, Mastra and genkit object tools already matched the existing patterns. - Tool detection: the reach walk and chunked re-scans recognise these shapes too.
- Scale handling: generated code is excluded, along with files over 400 KB and files over 150 KB with more than 4,000 lines. Semgrep uses all CPU cores.
These changes apply to every scan, so re-running experiment 1 today would not exactly reproduce the original run.
target list clone (atomic, checkpointed)
exp 1: npm funnel ──► <work>/src/<slug>
exp 2: curated frameworks
│
▼
┌────────────────────────────────────────────────────────────┐
│ src/scan.py — per repo │
│ │
│ Semgrep taint (3 tiers) run 2–3× and union │
│ ├ taint : tool-handler params (beats nondeterminism) │
│ ├ helper : any fn param + chunk re-scan every │
│ └ audit : any sink arg tool reg / switch case│
│ (beats fixpoint │
│ timeout) │
│ + shadow .ts tree for │
│ .mts/.cts │
│ │ │
│ ▼ │
│ reach walk: name-based call graph, handler → helper → │
│ sink (≤5 hops, skips tests/generic names) │
│ │ │
│ ▼ │
│ context demotion (auto_notes): fixed host, path │
│ containment, shell escaping, identifier allowlist │
└────────────────────────────────────────────────────────────┘
│
▼
PRIORITY QUEUE ──► manual triage ──► dedup (GHSA/issues/PRs,
(summary.md, (is it reachable? canonical+parent repo)
findings.csv) intended capability? │
confinement bypass?) ▼
sandboxed PoC (bwrap,
benign payload, local)
│
▼
coordinated disclosure
The scanner finds candidates; a human decides verdicts. Priority rank is a triage input, not a bug count.
For how each step decides what to surface, see research/METHODOLOGY.md. research/findings/exp1-mcp/pipeline.pdf is a plain-language walkthrough written for experiment 1, so it covers MCP servers only and uses the old project name and paths.
| path | holds |
|---|---|
src/ |
the Python entry points, described below |
tools/ |
rule generator, funnel filter, batch runner, PoC sandbox |
rules/, rules-audit/, tests/ |
generated Semgrep rules and their fixtures |
bench/ |
recall benchmark against published MCP bugs |
research/METHODOLOGY.md |
decision gates at each pipeline step, and the write-up of experiment 2's PoCs |
research/targets/ |
experiment 1 lists and funnel.csv; experiment 2 frameworks-*.txt and exp2_batch/ batch lists |
research/findings/exp1-mcp/ |
experiment 1 batch-NN/ verdicts and disclosure packages, plus SUMMARY.md and the pipeline PDF |
research/findings/exp2-frameworks/ |
experiment 2 batch-NN/ verdicts and PoCs, plus adhoc/ notes |
docs/ |
disclosure template |
architecture.drawio |
architecture diagram |
src/
src/scan.py: clones targets, runs the tiered Semgrep scan with reach walk and context demotion, and writesfindings.csv,repos.csvandsummary.md.src/hunt.py: runsscan.pyon one local directory orowner/repo, prints the priority queue and prior public reports, and savesreport.md.src/discover.py: buildsrepos.txt, a target list of mid-tier MCP servers from GitHub and npm search.
| check | result |
|---|---|
Rules (make test) |
21/21 against annotated positive/negative fixtures, including the agent-framework shapes |
Recall (make recall) |
5 published MCP bugs re-found and ranked at their vulnerable versions: CVE-2025-54073 (mcp-package-docs), CVE-2025-53355 (mcp-server-kubernetes), CVE-2025-5277 (aws-mcp-server, 3-hop chain), CVE-2025-53967 (figma-developer-mcp, 4-hop chain through a generic-named method), GHSA-v7q6-5mpg-q47w (crewly) |
| Patched versions | manual: the 4 CVE repos at HEAD have zero command-injection priority hits |
The recall benchmark contains only MCP servers, so agent-framework detection is covered by the fixtures alone. Scanning crewly takes up most of its runtime, about 18 minutes on its own. Read Known limits before trusting a clean result.
pip install semgrep pyyaml # semgrep 1.x runs offline; pyyaml is only for `make rules`
Python 3.10+, git. Clones, checkpoints and scan output stay out of the repo by default:
src/hunt.py writes under $FRAMEWORK_HUNTER_WORK/hunt/<slug> (default ~/framework-hunter-work),
tools/batch.py under ~/framework-hunter-work/scan. src/scan.py on its own writes to
./hunter-scan (gitignored); change it with --work. Set FRAMEWORK_HUNTER_VENV if Semgrep lives
in a virtual environment.
Run these from the repo root.
One target:
python3 src/hunt.py ./path/to/target # or: python3 src/hunt.py owner/repo
Prints the priority queue with reach chains, what was demoted and why, scan health, and (for
GitHub repos) prior public advisories/PRs. Saves report.md under
$FRAMEWORK_HUNTER_WORK/hunt/<slug>. For a framework, its "MCP server code: NOT detected" line is
expected.
A batch of agent frameworks (experiment 2):
python3 tools/batch.py 1 --list research/targets/frameworks-reversed.txt \
--work ~/framework-hunter-work/exp2 # batch 1 = the 10 least-used frameworks
Check a repo's size before scanning it. No code enforces the scale limit: agent-native (about
20k files) was skipped by hand and mastra was run alone.
A batch of MCP servers (experiment 1):
python3 src/discover.py -o repos.txt # GitHub + npm, mid-tier MCP servers
python3 tools/filter_tools.py <list> \
--src ~/framework-hunter-work/scan/src -o targets.txt --csv funnel.csv # keep only MCP servers with tools
tools/run_batches.sh 1 15 # scan in resumable batches of 10
src/discover.py and tools/filter_tools.py are MCP-specific. tools/batch.py and
tools/run_batches.sh read research/targets/repos-top200-downloads-tools.txt unless you pass
tools/batch.py --list.
Then open summary.md / findings.csv, work the priority queue top-down, and for each
candidate run the triage checklist below. Record verdicts in
research/findings/<experiment>/batch-NN/verdicts.md.
Semgrep OSS taint is intraprocedural (it can't follow a call from a handler into a helper), so the scan layers three source definitions and then reconnects them with the reach walk:
| tier | taint source | catches | noise |
|---|---|---|---|
| taint | registered tool-handler params: MCP (server.tool, registerTool, addTool, setRequestHandler, @mcp.tool, …) and agent frameworks (@tool, @function_tool, BaseTool._run, DynamicTool, …) |
bugs in the handler itself | low |
| helper | any function parameter | handler → helper → sink (all 4 CVE recall targets; figma is a 4-hop chain) | medium, filtered by reach_hint |
audit (--audit) |
any non-literal sink argument | custom tool registration the others miss | high |
Priority queue = taint findings + helper findings inside a tool file or ≤4 hops from one,
minus context-demoted rows. Demoted rows stay in findings.csv — verify before discarding.
- The sink value really comes from a tool argument or model output (follow
reach_hintby hand). - No validation between source and sink (allowlist, regex,
resolve+startsWith, quoting). - Is this the tool's intended capability? A
run_shelltool running shell, or a file tool reading files, is a design choice — unless it claims a sandbox, allowlist, or read-only mode you can bypass, or escapes a configured root. That bypass is the bug. - Threat model: stdio-local servers and framework tools matter via prompt injection of the agent; HTTP/SSE without auth is directly reachable. State which.
- Dedup first: check GHSA, issues and PRs on the repo and its parent/upstream before building a PoC. Most real bugs in popular targets are already public.
- PoC on your own clone only, benign payloads, in the sandbox (
tools/sandbox.sh).
- Clone and analyse locally only — never probe anyone's deployed instance.
- The scanner parses code; it never installs or runs it. PoCs go in bubblewrap/VM
(
npm installruns third-party scripts). - Coordinated disclosure: private report, ~90 days, publish after a fix. Keep this repo
private while
research/findings/has unreported or embargoed findings.
summary.md— counts, priority queue with GitHub permalinks, per-repo table.findings.csv— the dataset, one row per candidate. Auto columns (repo, commit, tier, rule, CWE, reach_hint, auto_notes, file:line, sink, permalink) plus triage columns you fill (reachable,sanitized_or_contained,poc_confirmed,status,cve, …).repos.csv— per-repo commit, tool count, findings per tier, scan health.research/findings/— per-batch verdicts and disclosure packages.
Semgrep OSS's taint fixpoint has a per-function time budget with no CLI knob. When a large
createServer() or switch(name) body exceeds it, every taint finding in that function is
dropped silently — the "Fixpoint timeout" line shows only under --debug — and the result set
is nondeterministic near the budget. This toolkit measures and works around it: 2–3× unioned
runs, and a chunked re-scan of each tool registration / switch case as its own small file so
each handler fits inside one budget. Quantified by the unstable_results, chunks and
chunk_only_findings columns.
Rules are generated from tools/gen_rules.py:
make rules && make test && make recall
Add a fixture (// ruleid: / // ok:) for every new sink, tool shape or false-positive fix. When
you confirm a new bug, add its repo + vulnerable commit to bench/recall.sh so it becomes a
regression test, once the bug is public.
- Not a soundness guarantee — a clean scan is not a clean target.
- Tool shapes outside the known MCP and framework patterns fall to the helper/audit tiers.
reach_hintis name-based, not a real call graph: same-named functions create false links, dynamic dispatch (handlers[name](args)) breaks real ones.- Containment/escaping is flagged (
auto_notes), not modelled in the rules. _run,_arunandforwardmatch by method name, so they can flag methods outside tool classes.- The large-file filter also drops hand-written files. Chunked re-scans still cover tool registrations, but a sink in a big helper file outside a registration can be missed.
- Most logic and classification bugs need manual review: the experiment 2 findings were an
evalfallback, a missing path separator and a missing DuckDB flag. - Languages: JS/TS and Python only. Go/Rust/C#/Java targets are out of scope.