awesome-llmsecops

Secure LLM and RAG deployment practices

Artificial General Intelligence Security discussions

Stream of AI attack examples and threat intelligence

Global AI security hub: curated research, articles, reports and tools

AI Security Operations: monitoring, incident response, SIEM/SOC integrations

Laboratory by Raft x ITMO University: breaking and defending AI systems

Open-source, AARM-aligned runtime security for AI agents: analyzes tool calls and reasoning traces, then detects/blocks prompt injection, malicious tool use, and out-of-remit actions in-flight (audit or block mode). LangChain/LangGraph/OpenAI Agents SDK; self-hostable offline.

A Comprehensive Benchmark to Evaluate LLMs as Agents (ICLR'24)

Guide for attack langchain agents

Full-stack benchmark for privacy leakage in multi-agent systems. Monitors 7 channels including tool calls, RAG queries, and inter-agent messages.

Official OWASP runtime defense layer that screens every read/write to AI agent memory, blocking prompt injection, secret leakage, and memory poisoning (ASI06). Integrations for LangChain, LlamaIndex, CrewAI, AutoGen.

Repello AI's CLI for extracting agentic workflows from LangChain/LangGraph/CrewAI/AutoGen and running automated threat modeling.

Practical examples of code for your best mlsecops pipeline

🔬 In-depth exploration of security theories, laws, terms and practices in Generative AI

GitHub repository on RAG attacks, mitigations, and defense strategies

Collection of RAG-based LLM attack techniques

Article about RAG vulnerabilities

Blog post about RAG poisoning and its effects on LLaMA3

GitHub repository about poisoned RAG systems

Article on security risks in Retrieval-Augmented Generation (RAG)

Research paper on security vulnerabilities in agentic browsers

Analysis of security risks in browser-based AI agents

Brave's analysis of prompt injection vulnerabilities in Perplexity Comet browser

Research on zero-click Google Drive wiper attack via Perplexity Comet. Shows how polite, well-structured emails can trigger destructive actions in agentic browsers.

Benchmark for agent security

Comprehensive benchmark for AI safety evaluation

Research paper on AI safety benchmarking methodologies

Human-grounded benchmark for testing AI agent security. Built by Lakera with UK AI Security Institute using 194,000+ human attack attempts from Gandalf: Agent Breaker. Tests backbone LLM resilience across 10 threat snapshots.

Research paper on the Backbone Breaker Benchmark methodology and findings

Benchmark of five skill scanners (NVIDIA SkillSpector, VirusTotal, ClawScan, static analysis) on 60 manually labeled ClawHub skills — 20 benign, 20 vulnerable, 20 malicious — across code-based and code-free attack vectors. Labeled dataset public.

Interactive game focusing on AI security challenges

AI CTF competition from PHDFest2 2025

Russian platform for AI security training

Challenges to test your knowledge of AI, ML, and LLMs

Free interactive learning lab for AI/LLM/agentic security. 91 sourced real-world incident cases with root-cause and architecture walkthroughs, 34 hands-on attack-scenario simulations, risk taxonomy cross-mapped to OWASP LLM Top 10 / MITRE ATLAS, and a preventive/detective/corrective control library.

Free LLM security testing

Open-source LLM evaluation framework with 50+ metrics, LLM-as-Judge augmentation, and guardrail scanners (jailbreak, PII, prompt-injection); AutoEval pipelines with CI/CD support.

Open-source self-hostable end-to-end agent engineering and optimization platform unifying tracing, evaluation, simulation, datasets, gateway, and guardrails in one feedback loop.

LLM monitoring and security platform for GenAI applications. Detects prompt injection, jailbreaks, malicious HTML/Markdown elements, and PII. Provides real-time anomaly detection and security alerts.

Open Source LLM Engineering Platform with security capabilities.

AI monitoring dashboard for AI agents and LLMs. Demo

Open-source platform for LLM observability, evaluations, and prompt optimization.