awesome-agent-failures

Analyzer/Verifier agent pair entered an undetected feedback loop for 264 hours (11 days), accruing $47K in API costs with no useful output; observability without enforcement.

Counsel's unverified AI research reached two expert reports, which carried identical fabricated citations; the court excluded an expert opinion as inherently unreliable under Rule 702, then dismissed the claims with prejudice. The first case study here where the fabrications themselves cost the client the case.

Airline held liable after chatbot gave incorrect bereavement fare information, ordered to pay $812 in damages.

Amazon Q gave engineers guidance from an outdated wiki, causing four high-severity incidents in one week, 6.3M lost orders, and a six-hour customer-facing outage.

Attacker injected prompt into official AWS extension telling Amazon Q to delete filesystems and wipe S3 buckets; only a syntax error prevented mass destruction across 1M+ installs.

Senior AI reporter Benj Edwards fired after using a Claude Code–based extraction tool that fabricated quotes attributed to engineer Scott Shambaugh; article retracted, called "a serious failure of our standards."

Classifies multi-agent research failures along severity × fixability × visibility × capability locus, drawn from 1000+ iterations across two flagship deployments.

Conference on AI agent risks.

Vectara's analysis of RAG hallucination challenges.

Explores the risks of advanced AI and argues for aligning AI systems with human values to ensure safety.

Investigates how AI systems inherit human biases and examines efforts to align machine learning with ethical and social values.

Comprehensive survey of LLM-based agents.

Share experiences and ask questions.

Help us improve this resource.

Get notified of new failure patterns.