awesome-agent-failures

Analyzer/Verifier agent pair entered an undetected feedback loop for 264 hours (11 days), accruing $47K in API costs with no useful output; observability without enforcement.

Airline held liable after chatbot gave incorrect bereavement fare information, ordered to pay $812 in damages.

Amazon Q gave engineers guidance from an outdated wiki, causing four high-severity incidents in one week, 6.3M lost orders, and a six-hour customer-facing outage.

Attacker injected prompt into official AWS extension telling Amazon Q to delete filesystems and wipe S3 buckets; only a syntax error prevented mass destruction across 1M+ installs.

Senior AI reporter Benj Edwards fired after using a Claude Code–based extraction tool that fabricated quotes attributed to engineer Scott Shambaugh; article retracted, called "a serious failure of our standards."

Tasked with indexing a small hobbyist network, an agent deployed five 48-vCPU AWS instances with no cost preview, running up a bill of $6,531 after an operator approved "immediately without delay" with no plan review.

Classifies multi-agent research failures along severity × fixability × visibility × capability locus, drawn from 1000+ iterations across two flagship deployments.

Conference on AI agent risks.

Vectara's analysis of RAG hallucination challenges.

Investigates how AI systems inherit human biases and examines efforts to align machine learning with ethical and social values.

Explores the risks of advanced AI and argues for aligning AI systems with human values to ensure safety.

Comprehensive survey of LLM-based agents.

Share experiences and ask questions.

Help us improve this resource.

Get notified of new failure patterns.