Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents
by Firecrawl
Turn the live web into LLM-ready markdown — search, scrape, crawl, click through and parse, behind one API your agent can call.
Overview
Firecrawl's documentation covers a v2 API summarised on its own front page as 'Search the web, scrape any page, and interact with it, all through one API.' The endpoint set is what makes it a single dependency rather than a scraper: Scrape converts one URL to markdown, HTML, screenshots or schema-constrained JSON; Search runs a live web query and returns full page content rather than snippets; Crawl walks an entire site; Map enumerates a site's URLs; Parse converts local PDFs, DOCX, XLSX and HTML into the same formats; Interact clicks buttons and fills forms to reach content behind interaction; Agent gathers data autonomously; and a managed Browser Sandbox plus webhooks and batch scraping handle stateful and asynchronous jobs. Official SDKs cover Python, Node.js, Go, Java, Rust, Ruby, Elixir, .NET and PHP alongside a CLI, and there are documented integrations for LangChain, LlamaIndex, OpenAI, the ChatGPT and Claude.ai connectors, and an MCP server so a coding agent can call it directly. The core engine is open source under AGPL-3.0 (SDKs and UI components are MIT) with a self-hosting guide, and the repository is one of the most-starred in the space at roughly 170k stars. The hosted service is metered in credits: 1,000 free per month, then Hobby at $16/month for 5,000 pages, Standard $83, Growth $333 and Scale $599, with scrape, crawl and map costing one credit per page.
At a Glance
- Topic
- Agentic
- Level
- All Levels
- Format
- Documentation
- Cost
- Freemium
- Duration
- ~2-3 hours for the core endpoint guides, then ongoing reference
- Provider
- Firecrawl
- Hands-on
- Yes — code/exercises
- Certificate
- None
What You’ll Learn
- ✓Convert any URL into clean, token-efficient markdown instead of raw HTML for LLM input
- ✓Extract structured JSON from a page by supplying a schema rather than writing selectors
- ✓Run live web searches that return full page content for grounding agent answers
- ✓Crawl and map an entire site to build a corpus for a RAG index
- ✓Drive interactive and authenticated pages by clicking, filling forms and holding browser sessions
- ✓Parse local PDFs, DOCX and XLSX through the same pipeline as web pages
- ✓Wire Firecrawl into LangChain, LlamaIndex or a coding agent via its MCP server
- ✓Handle long jobs asynchronously with batch scraping and webhook callbacks
Highlights
- •One API replaces the usual stack of proxy rotation, headless browser and per-site parser code
- •Search returns full page content, not snippets — the difference between a demo RAG agent and a useful one
- •Nine official SDKs plus a CLI and an MCP server, so it drops into Python, TypeScript or a coding agent unchanged
- •Core engine is genuinely open source (AGPL-3.0) with a self-hosting path, and is among the most-starred repos in AI tooling at ~170k stars
- •Honest caveat from the community: self-hosting the full stack is heavy enough that lighter forks and rewrites exist (Firecrawl-Simple, and the Rust 'Draco' project posted to Hacker News in August 2026), and the hosted tiers get expensive at volume — 1,000 free credits per month is a small budget for a crawl
- •Documentation ships llms.txt and llms-full.txt so your own agent can read the API reference
Who It’s For
Best For
- ✓Engineers building RAG pipelines that need fresh, clean web content
- ✓Teams building research or deep-search agents that must read the live web
- ✓Data engineers replacing bespoke scraper maintenance with a single API
- ✓Developers who want web access inside a coding agent via MCP
Prerequisites
- •Comfortable calling a REST API from Python or JavaScript and handling async jobs
- •A Firecrawl API key for the hosted service, or Docker experience if self-hosting
- •Basic understanding of how retrieval feeds an LLM prompt
FAQ
What is Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents?
The official documentation for Firecrawl, an open-source web data API that converts arbitrary web pages into clean markdown or structured JSON for LLM and agent pipelines. It is for engineers building RAG systems, research agents or scrapers who are tired of writing per-site parsers, rotating proxies and rendering JavaScript by hand. You finish able to search the live web, extract structured fields from any page, crawl a whole site, and drive authenticated flows from code.
Is Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents free?
Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents offers free content, with paid options for certificates or premium features.
What level is Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents for?
Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents is aimed at a all levels audience. Recommended background: Comfortable calling a REST API from Python or JavaScript and handling async jobs, A Firecrawl API key for the hosted service, or Docker experience if self-hosting, Basic understanding of how retrieval feeds an LLM prompt.
How long does Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents take?
Expect roughly ~2-3 hours for the core endpoint guides, then ongoing reference. Most learners work through it at their own pace.
What will I learn from Firecrawl Documentation — Web Search, Scraping and Browsing for AI Agents?
You'll learn: Convert any URL into clean, token-efficient markdown instead of raw HTML for LLM input; Extract structured JSON from a page by supplying a schema rather than writing selectors; Run live web searches that return full page content for grounding agent answers; Crawl and map an entire site to build a corpus for a RAG index; Drive interactive and authenticated pages by clicking, filling forms and holding browser sessions; Parse local PDFs, DOCX and XLSX through the same pipeline as web pages; Wire Firecrawl into LangChain, LlamaIndex or a coding agent via its MCP server; Handle long jobs asynchronously with batch scraping and webhook callbacks.
Topics
Sources
This page was written from 3 sources, 2 on domains other than docs.firecrawl.dev.