awesome-security-for-ai

Automates the tasks of stress-testing, red-teaming, and evaluating your AI systems for critical failure.

Continuous AI red teaming for LLMs.

Platform for testing and launching LLM applications securely.

A LLM vulnerability scanner.

Scan models for vulnerabilities and supply chain issues.

Python package for language model jailbreak evaluation.

Research roundup on AI's use in classic security tools.

A curation of awesome tools, documents and projects about LLM Security.

An awesome list of papers on privacy attacks against machine learning.

Trail of Bits' machine learning security references, guidance, and tools.

AI security resources including attacks, examples, and code.

AI security related frameworks, attacks, tools and papers.

Provide visibility into an AI system, characterize attack surfaces, and assess vulnerabilities in an organization.

AI governance, risk, and compliance for the AI-powered enterprise.

Provides automated stress testing of AI systems and autogenerates documentation needed for regulatory audits.

Track projects and apply frameworks to them.

Use differential privacy and PII sanitization on training data.

Use existing data and the power of Generative AI for synthetic data generation.

"Stained glass transforms" of text and image data when training preserves privacy in model and inferences.

Simulation and synthetic data for computer vision training.

Identify / defend against any type of non-compliance as defined by your specific AI policies and catch attacks.

Guardrails runs Input/Output Guards in your application that detect, quantify and mitigate the presence of specific types of risks.

Proactively defend against threats to your LLMs.

Protection from prompt injections, data loss, and toxic content.

Detect, anonymize, and replace PII with less than half the error rate of alternatives.

SDK and proxy for protection against common prompt attacks.

Train encrypted models and do encrypted inferences over them.

Encrypt vector embeddings before sending to a vector database to secure the data in RAG workflows and other AI workflows.

Run AI models inside Intel SGX and other enclave technologies.

Test and track prompts, their performance and versions over time.

Open-source fuzzing framework specifically designed for LLMs, especially for their integrations in applications via APIs.

Open-source tool to help you harden your GenAI applications.