Security Tools
41Open-source and commercial tools for AI security testing
AI Runtime Security
Real-time detection and firewall platform for protecting AI models against adversarial attacks in production.
AI Discovery Module
Shadow AI and asset inventory tool for discovering unmanaged AI models across your organization.
AI Attack Simulation
Adversarial testing platform that simulates real-world attacks against AI models.
AI Supply Chain Security
Model integrity verification and supply chain security for AI weights and artifacts.
Agentic & MCP Protection
Defense platform for securing autonomous AI agents and MCP protocol communications.
PurpleLlama
Meta's open-source cybersecurity evaluation suite including CyberSecEval for LLM safety benchmarking.
PyRIT
Microsoft's Python Risk Identification Tool for generative AI red teaming and risk assessment.
NeMo Guardrails
NVIDIA's toolkit for adding programmable guardrails to LLM conversational systems.
promptfoo
Open-source LLM evaluation framework with built-in support for prompt injection and jailbreak testing.
garak
LLM vulnerability scanner that probes for hallucination, data leakage, prompt injection, and more.
LLM Guard
Input/output sanitization library for LLMs with PII detection and prompt injection filtering.
Guardrails AI
Structured output validation framework ensuring LLM responses conform to expected schemas.
P4RS3LT0NGV3 (Parseltongue)
Payload crafting and obfuscation tool for security testing and prompt injection research.
Agentic Radar
CLI security scanner for discovering vulnerabilities in agentic AI workflows.
MCP Scanner
Automated probe for identifying security weaknesses in MCP server deployments.
MCP Shield
Defensive gateway for neutralizing protocol-level adversarial requests targeting MCP servers.
Invariant
Trace analysis tool for detecting logic flaws and data leaks in AI agent interaction logs.
Prompt Fuzzer
Dynamic hardening tool that fuzzes LLM system prompts to discover injection vulnerabilities.
LLMFuzzer
Prompt generation fuzzing tool for automated discovery of LLM jailbreak vectors.
Spikee
Injection payload kit for testing LLM application resilience against diverse attack patterns.
Vigil
Risk scoring and detection API for evaluating prompt injection threats in real-time.
CipherChat
Encrypted LLM communication research tool exploring cipher-based safety bypass techniques.
BrokenHill
Automated GCG attack tool for generating adversarial suffixes that bypass LLM alignment.
WhistleBlower
System prompt inference tool that extracts hidden instructions from deployed LLM applications.
Splunk AI Assistant
AI-powered security analyst workflow tool integrated with Splunk's detection ecosystem.
OpenCRE-Chat
AI chatbot interface for querying and cross-referencing security standards and controls.
PromptLayer
Prompt management and observability platform with security monitoring capabilities.
MCP Safety Scanner
Automated protocol audit tool for scanning MCP server configurations and vulnerabilities.
Agent Security Scanner MCP
Security scanner for detecting SQLi, XSS, and secret exposure in AI agent deployments.
Team Xint AIxCC Entry
AI-based vulnerability hunting tool developed for the AIxCC competition.
L1B3RT45
AI jailbreaking toolkit for security researchers testing LLM safety boundaries.
Imperio
Prompt anchoring tool for research into instruction hierarchy and priority manipulation.
Last Layer
Low-latency pre-filter for detecting adversarial prompts before they reach the LLM.
LocalMod
Self-hosted content moderation tool for filtering harmful LLM outputs locally.
Prompt Shield (Action)
CI/CD GitHub Action for detecting indirect prompt injection in automated pipelines.
Jailbreak Evaluation
Python evaluation framework for measuring LLM safety against jailbreak attack methods.
LLM Confidentiality Tool
Tool for testing and preventing data leakage from LLM system prompts and context.
Virtual Prompt Injection
Virtual testing environment for researching prompt injection in sandboxed LLM contexts.
Ollama DEBUG Log Mod
Modified Ollama fork enabling prompt-inclusive debug logging for security monitoring.
Open Prompt Injection
Evaluation benchmark tool for measuring LLM resilience against prompt injection attacks.
Jailbreaking LLMs (PAIR)
Black-box jailbreak refinement tool using the PAIR algorithm for automated attack generation.
Know a resource we're missing?
This directory is community curated. Submit a pull request to add your favorite AI security resources.
Contribute on GitHub