Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
-
Updated
Nov 22, 2026 - Python
Open-source AI red teaming toolkit from Microsoft. Test generative AI systems with automated scans, adaptive multi-turn attacks, and a web UI.
Did your fine-tune just get less safe? Redline runs the same safety tests on two versions of your AI model and tells you in one plain sentence whether to ship. Try it in 2 minutes: pip install redline-evals && redline try
End-to-end Al/LLM security, safety and penetration testing framework for prompt injection, jailbreaks, RAG security, agentic attacks, red teaming, and monitoring.
Evaluation and observability lab for LLM agents: behavior, safety, latency, cost, redacted traces, and repeatable CI.
Test LLM safety rules in Arabic: refusal accuracy and over-refusal across MSA and 6 dialects. YAML cases, a CLI, and a CI gate.
AI agent for agentic AI safety testing and security gate validation, built with Agentic Star.
Deterministic Evals, Benchmarks and Safety Testing for LLM agents (Python 3.12, FastAPI, Pydantic v2, Pytest, LiteLLM)
DeepEval LLM evaluation framework using Claude API — tests GEval, Hallucination, Answer Relevancy, Faithfulness, Contextual Precision/Recall, Bias, Toxicity, PII Leakage, and Tool Correctness on a mock AI customer support assistant.
To associate your repository with the safety-testing topic, visit your repo's landing page and select "manage topics."