Skip to content

stress_test: add --synthetic cases for models missing from the set export - #11

Merged
m-messer merged 1 commit into
mainfrom
stress-test-smollm2
Nov 13, 2026
Merged

m-messer merged 1 commit into
mainfrom
stress-test-smollm2

Conversation

@m-messer

Copy link
Copy Markdown
Member

Summary

The Context engineering set export has no smollm2_infer response area, so scripts/stress_test.py couldn't load-test SmolLM2.

This adds:

  • --synthetic MODEL (repeatable): builds test cases from built-in student-style prompts (5 for smollm2_infer). When it's given, set-export cases are only included if --model is also passed.
  • --word-counts (default 16,64,256): each prompt is sent once per length. Each length gets its own latency row, since generation length drives latency.

Running without --synthetic works as before.

Staging results (2026-11-13)

python3 scripts/stress_test.py --synthetic smollm2_infer --ramp 1,5,10,25,50 --duration 60 --timeout 120
  • 1,134 of 1,137 requests succeeded. The 3 failures were 503 Service Unavailable at exactly 30s, all in the 5-user stage, most likely the gateway timing out while new instances started.
  • Median latency barely changes under load: about 0.8s / 2.5s / 9.5s for 16 / 64 / 256 tokens. Throughput rose from 0.32 to 4.44 req/s.
  • Slow outliers come from new instances starting up (cold starts): p99 reaches 68–73s at 25–50 users (max 87s).

Test plan

  • --synthetic smollm2_infer --dry-run prints 15 well-formed request bodies
  • --dry-run without --synthetic still loads the 11 set-export cases
  • Full ramp against staging (results above)

🤖 Generated with Claude Code

…port

The Context engineering set has no smollm2_infer response area, so the
script could not load-test it. --synthetic MODEL builds cases from
built-in student-style prompts crossed with --word-counts (default
16,64,256). Each length is reported as its own row, since generation
length drives latency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@m-messer
m-messer merged commit 0bd672b into main Nov 13, 2026
1 check passed
@m-messer
m-messer deleted the stress-test-smollm2 branch November 13, 2026 09:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant