All Resources

Security Tools

41

Open-source and commercial tools for AI security testing

AI Runtime Security

Real-time detection and firewall platform for protecting AI models against adversarial attacks in production.

real-time-detectionfirewallruntime-protection

AI Discovery Module

Shadow AI and asset inventory tool for discovering unmanaged AI models across your organization.

shadow-aiasset-inventorydiscovery

AI Attack Simulation

Adversarial testing platform that simulates real-world attacks against AI models.

adversarial-testingred-teamingsimulation

AI Supply Chain Security

Model integrity verification and supply chain security for AI weights and artifacts.

supply-chainmodel-integrityweights

Agentic & MCP Protection

Defense platform for securing autonomous AI agents and MCP protocol communications.

agentic-securitymcpautonomous-agents

PurpleLlama

Meta's open-source cybersecurity evaluation suite including CyberSecEval for LLM safety benchmarking.

cybersecurity-evalbenchmarkingmeta

PyRIT

Microsoft's Python Risk Identification Tool for generative AI red teaming and risk assessment.

red-teamingrisk-identificationmicrosoft

NeMo Guardrails

NVIDIA's toolkit for adding programmable guardrails to LLM conversational systems.

guardrailspolicy-controlhallucination

promptfoo

Open-source LLM evaluation framework with built-in support for prompt injection and jailbreak testing.

evaluationprompt-injectionjailbreak-testing

garak

LLM vulnerability scanner that probes for hallucination, data leakage, prompt injection, and more.

vulnerability-scannerprobingautomated-testing

LLM Guard

Input/output sanitization library for LLMs with PII detection and prompt injection filtering.

sanitizationpii-detectionprompt-injection

Guardrails AI

Structured output validation framework ensuring LLM responses conform to expected schemas.

validationstructured-outputguardrails

P4RS3LT0NGV3 (Parseltongue)

Payload crafting and obfuscation tool for security testing and prompt injection research.

payload-craftingobfuscationprompt-injection

Agentic Radar

CLI security scanner for discovering vulnerabilities in agentic AI workflows.

agentic-securityworkflow-scanningcli

MCP Scanner

Automated probe for identifying security weaknesses in MCP server deployments.

mcpprotocol-securityscanning

MCP Shield

Defensive gateway for neutralizing protocol-level adversarial requests targeting MCP servers.

mcpprotocol-defensegateway

Invariant

Trace analysis tool for detecting logic flaws and data leaks in AI agent interaction logs.

trace-analysislogic-flawsdata-leaks

Prompt Fuzzer

Dynamic hardening tool that fuzzes LLM system prompts to discover injection vulnerabilities.

fuzzingprompt-injectionhardening

LLMFuzzer

Prompt generation fuzzing tool for automated discovery of LLM jailbreak vectors.

fuzzingjailbreakprompt-generation

Spikee

Injection payload kit for testing LLM application resilience against diverse attack patterns.

payload-kitinjection-testingresilience

Vigil

Risk scoring and detection API for evaluating prompt injection threats in real-time.

risk-scoringdetection-apiprompt-injection

CipherChat

Encrypted LLM communication research tool exploring cipher-based safety bypass techniques.

encryptionsafety-bypassresearch

BrokenHill

Automated GCG attack tool for generating adversarial suffixes that bypass LLM alignment.

gcg-attacksadversarialalignment-bypass

WhistleBlower

System prompt inference tool that extracts hidden instructions from deployed LLM applications.

system-promptinferenceextraction

Splunk AI Assistant

AI-powered security analyst workflow tool integrated with Splunk's detection ecosystem.

siemanalyst-workflowdetection

OpenCRE-Chat

AI chatbot interface for querying and cross-referencing security standards and controls.

standardschatbotcontrol-mapping

PromptLayer

Prompt management and observability platform with security monitoring capabilities.

prompt-managementobservabilitymonitoring

MCP Safety Scanner

Automated protocol audit tool for scanning MCP server configurations and vulnerabilities.

mcpprotocol-auditscanning

Agent Security Scanner MCP

Security scanner for detecting SQLi, XSS, and secret exposure in AI agent deployments.

sqlixsssecret-detection+1

Team Xint AIxCC Entry

AI-based vulnerability hunting tool developed for the AIxCC competition.

vulnerability-huntingai-competitionautomated

L1B3RT45

AI jailbreaking toolkit for security researchers testing LLM safety boundaries.

jailbreakingsafety-testingred-teaming

Imperio

Prompt anchoring tool for research into instruction hierarchy and priority manipulation.

prompt-anchoringinstruction-hierarchyresearch

Last Layer

Low-latency pre-filter for detecting adversarial prompts before they reach the LLM.

pre-filterlow-latencydetection

LocalMod

Self-hosted content moderation tool for filtering harmful LLM outputs locally.

moderationself-hostedcontent-filtering

Prompt Shield (Action)

CI/CD GitHub Action for detecting indirect prompt injection in automated pipelines.

ci-cdgithub-actionprompt-injection

Jailbreak Evaluation

Python evaluation framework for measuring LLM safety against jailbreak attack methods.

evaluationjailbreaksafety-measurement

LLM Confidentiality Tool

Tool for testing and preventing data leakage from LLM system prompts and context.

confidentialityleakage-preventiontesting

Virtual Prompt Injection

Virtual testing environment for researching prompt injection in sandboxed LLM contexts.

virtual-testingprompt-injectionsandbox

Ollama DEBUG Log Mod

Modified Ollama fork enabling prompt-inclusive debug logging for security monitoring.

ollamadebug-loggingmonitoring

Open Prompt Injection

Evaluation benchmark tool for measuring LLM resilience against prompt injection attacks.

evaluationbenchmarkprompt-injection

Jailbreaking LLMs (PAIR)

Black-box jailbreak refinement tool using the PAIR algorithm for automated attack generation.

jailbreakpair-algorithmblack-box

Know a resource we're missing?

This directory is community curated. Submit a pull request to add your favorite AI security resources.

Contribute on GitHub