Skip to main content

🔥 A list of tools, frameworks, and resources for building AI web agents

1.5k
GitHub Stars
105
Curated Resources
7
Categories
6 hours ago
Last Refreshed
Autonomous Web AgentsAI Web Automation ToolsAI Web Scrapers/CrawlersWeb Search & Query ToolsBenchmarks & ResearchTutorials & GuidesJoin the Community

Use this list with your AI agent

Add the Context Awesome MCP server to Claude, Cursor, or any MCP client, then ask:

"Show me computer-use agents resources from awesome-web-agents"

Installation instructions →

What's inside

Autonomous Web Agents

  • Agent-E

    Agent & framework with HTML DOM distillation.

  • AgentGPT

    Deploy autonomous AI agents in your browser.

  • Anthropic Computer UseComputer-use Agents

    Computer use agent that can control your browser.

  • Automina

    AI browser automation tool with natural language control.

  • Browserable

    An open-source & self-hostable browser automation library for AI agents.

  • Browser-Use

    SOTA agent and framework that makes the web LLM-friendly.

AI Web Automation Tools

  • agent-qa

    Self-improving QA harness for natural-language web test runs with execution memory and self-healing actions.

  • Asteroid.ai

    Hosted browser agents for SMEs to automate complex workflows.

  • Autobrowser

    A free, experimental Chrome extension that leverages Claude Computer Use to automate tasks in your browser.

  • AutoGPTDev Tools

    Experimental agent for task completion and web browsing.

  • BabelWrapDev Tools

    HTTP API and MCP server that lets AI agents interact with websites through natural language instead of CSS selectors.

  • Bardeen.ai

    A Chrome extension that enables AI-powered browser automations, allowing users to automate tasks and workflows directly within the browser.

Web Search & Query Tools

  • AgentQL

    A query language and toolkit that makes the web AI-ready.

  • Exa.ai

    The fastest and most accurate web search API for AI agents.

  • Jina.ai

    Neural search platform for web data.

  • Not Human Search

    Search engine that indexes 1,750+ agent-first tools ranked by agentic readiness. Available as an MCP server with tools for searching, scoring, and monitoring agent infrastructure.

  • SerpAPI

    Search API that provides Google Search results for your agents.

  • Serper.dev

    Performant and cost effective search API that provides Google Search results for your agents.

Benchmarks & Research

  • Bananalyzer

    An open-source evaluation framework for web-based AI agents.

  • BrowserGym by ServiceNow

    A gym environment for web task automation.

  • ClawBench

    283 everyday tasks (V1 153 + V2 130) on 163 live production websites across 15 categories. Two-stage scoring (final HTTP-request interception + LLM judge) blocks only the write request so real sites stay clean. Public leaderboard with 5-layer execution traces (recording, action log, request log, agent messages, interception).

  • Mind2Web

    A large-scale dataset for generalist web agents.

  • MiniWoB++

    A classic suite of 104 mini web browser tasks in a synthetic environment. It is an extension of the OpenAI MiniWoB benchmark.

  • TimeWarp

    A benchmark on historical versions of web UI.

Tutorials & Guides

AI Web Scrapers/Crawlers

  • Crawl4AI

    Open-source LLM Friendly Web Crawler & Scraper.

  • DataLens

    AI web data agent that plans browser scraping tasks, extracts structured web data, and exposes MCP tools for dataset workflows.

  • Expand.ai

    Turns any website into a type-safe API you can rely on.

  • FireCrawl

    APIs for turning websites into LLM-friendly markdown.

  • LLM Scraper

    Uses LLMs for intelligent scraping and content understanding.

  • Plasmate

    Open-source headless browser engine for AI agents. Compiles HTML to Semantic Object Model (SOM) with 17.5x token compression. 13 MCP tools. First browser tool on the MCP Registry. Rust, Apache-2.0.

Showing a sample of 105 resources. View the full list on GitHub →