AI workers for Firecrawl

What is an AI worker for Firecrawl?

An AI worker for Firecrawl is an AI agent scoped to one web-reading job, such as watching a competitor's pricing page or turning a documentation site into something it can answer questions from. It reaches the public web through your connected Firecrawl account, and where it may go, what it may bring back and how much it may spend are all settled before it runs.

What these workers do

Watch the pages that matter

Reads a handful of competitor or supplier pages on a schedule and reports what changed in the wording, the pricing or the claims since last time.

Turn a documentation set into something answerable

Crawls a docs site into clean text overnight, so a worker can answer questions from it instead of guessing from a homepage.

Pull fields out of pages that have no API

Describes the fields you want and returns them as structured data across a set of pages, ready to compare week on week.

Research a question across the open web

Searches, then reads the results that look worth reading, and comes back with the sources rather than an unattributed summary.

How an AI worker runs in Firecrawl

The reach is the open web, so scope is the first question

A worker allowed only a section of a site can read those pages but cannot crawl the whole host, because a crawl fans out across pages that rule never covered. That distinction is enforced rather than advised.

Reading is cheap, collecting is not, and they are separate switches

You choose what comes back, and that decides the bill

The powers that reach past the firewall stay off

What web reading is actually for

Firecrawl questions, answered.

Can a worker read any site on the internet?

Only the ones you allow. Because the reach is so broad by nature, this connection is normally scoped to named sites, and the check happens before the request is sent rather than being an instruction the worker is trusted to follow.

How do we stop a worker running up a large bill?

The jobs that cost real money — crawling a site, scraping a batch of URLs, running an extraction — sit behind their own switch, separate from permission to read a page. A worker can be given reading without spending, and each job can be capped at a number of pages.

Can one worker read another worker's results?

No. A crawl or extraction runs in the background and is collected afterwards, and each is tied to the worker that started it. Another worker asking for the same job is told it does not exist, even on the same account.

Does it change anything on the sites it visits?

No. Everything here is reading: pages, search results and extracted fields. Nothing is submitted or altered on any site, and results come back through the same firewall that decided what could be reached.

What AI workers can do in Firecrawl

The tools a worker can be granted in Firecrawl once you connect it. A kit grants only the ones its job needs, and you can narrow them further on the worker.

Firecrawl Scrape Page

Fetch one URL through Firecrawl and get it back as clean markdown, plus whichever other formats you ask for (`summary`, `links`, `html`, `rawHtml`, `screenshot`, `json`). Use it to read a page verbatim, including sites that only render under JavaScript — `options.waitFor` for slow pages, `options.onlyMainContent` to drop navigation and boilerplate. The URL is checked against this connection's egress (domain/URL allow-block) rules BEFORE anything is fetched or billed, so a refusal is a rule rather than a broken page. Read `accessInfo`: unavailable formats are dropped only when at least one permitted format remains; an all-unavailable request is refused before fetching or billing, so a missing `screenshot`, `html` or `json` means the format was removed, not that the page had none.

Firecrawl Map Site

Discover the URLs of a site without scraping any of them: give a seed URL and get back the links Firecrawl can find, optionally ordered by relevance to `search`. Fast and cheap compared with a crawl — use it to plan one, or to locate the handful of pages actually worth scraping with firecrawl_scrape. Discovered links are filtered by this connection's egress (domain/URL allow-block) rules, so read `accessInfo`: a short list can be scope rather than a small site.

Firecrawl Web Search

Search the web through Firecrawl and get ranked results with title, URL and snippet. Pass `scrapeOptions` to also pull each result's page content inline in the same call — convenient, but it scrapes every result and costs credits per page, so raise `limit` deliberately. Narrow with `sources` (web, images, news), `tbs` for recency ('qdr:d' day, 'qdr:w' week, 'qdr:m' month), and `location`/`country` for locale. Result URLs are filtered by this connection's egress (domain/URL allow-block) rules and inline formats this token cannot see are dropped, so read `accessInfo` before treating the list as exhaustive.

Firecrawl Crawl Site

Start a whole-site crawl from a seed URL: Firecrawl discovers pages and scrapes each one. This call is ASYNCHRONOUS and it SPENDS CREDITS — it returns a job `id` and no content at all, so poll firecrawl_get_job with that id until `status` is 'completed' (or 'failed'/'cancelled') and read the documents from there; firecrawl_cancel_job stops it early. Every delivered page costs a credit, so narrow before you run: `limit` caps pages, `includePaths`/`excludePaths` restrict which paths are visited, `maxDiscoveryDepth` limits how far links are followed. The page budget is clamped to this token's per-job cap, off-domain options are ignored unless this token is allowed to leave the seed host, and every crawled URL is checked against this connection's egress (domain/URL allow-block) rules — read `accessInfo` on this call and on each poll, since dropped formats and filtered URLs are reported there.

Firecrawl Batch Scrape

Scrape many URLs you already know in one job — the bulk form of firecrawl_scrape, for a list you assembled from firecrawl_map or firecrawl_search. This call is ASYNCHRONOUS and it SPENDS CREDITS: it returns a job `id` (plus any `invalidUrls` Firecrawl rejected up front) and no page content, so poll firecrawl_get_job with that id until `status` is 'completed' to read the documents; firecrawl_cancel_job stops it early. Each delivered page costs a credit. Every URL is checked against this connection's egress (domain/URL allow-block) rules before it is fetched or billed, and output formats this token cannot see are dropped from the request — read `accessInfo` here and on each poll, because a shorter document list than URL list is usually scope rather than failure.

Firecrawl Extract Data

Pull structured records out of one or more pages — give the URLs (wildcards like 'https://example.com/blog/*' are allowed) plus a `prompt` and/or a JSON `schema`, and Firecrawl returns rows shaped to it instead of raw page text you have to parse. This call is ASYNCHRONOUS and it SPENDS CREDITS: it returns a job `id`, so poll firecrawl_get_job with that id until `status` is 'completed' to read the extracted data. Extract jobs cannot be cancelled once started, so keep the URL set tight. The output rides the `json` field, which a token can be denied — read `accessInfo`, since source URLs are also filtered by this connection's egress (domain/URL allow-block) rules and `enableWebSearch` is ignored unless this token may reach beyond the URLs you named.

Firecrawl List Jobs

List the asynchronous Firecrawl jobs this token started — crawls, batch scrapes and extractions — newest first, each with its `id`, `jobType`, seed URL and status. Use it to recover a job id you did not keep, then read the results with firecrawl_get_job. Jobs started by anyone else are never listed, so an empty list means this token launched nothing, not that the account is idle. Read `accessInfo` for anything else that limited the view.

Firecrawl Get Job

Poll one asynchronous job — a crawl, batch scrape or extraction — by the `jobId` its launch returned. Gives `status` ('scraping' while it runs, then 'completed', 'failed' or 'cancelled'), progress as `completed`/`total`, `creditsUsed`, and every document that has landed so far; keep polling while the status is 'scraping'. `hasMore` true means more result pages exist upstream than were followed on this call. Only jobs this token started are readable and an unknown id is reported as not found. Documents are filtered by this connection's egress (domain/URL allow-block) rules and masked to the fields the job was launched with, so read `accessInfo` before concluding a page came back empty or a format was unavailable.

Firecrawl Cancel Job

Cancel a running crawl or batch-scrape job by `jobId` so it stops consuming credits — worth doing as soon as a job is clearly going wider than intended. Pages already delivered stay readable with firecrawl_get_job, but pages not yet fetched are abandoned and a cancelled job cannot be resumed; start a narrower job instead. Extraction jobs cannot be cancelled. Only jobs this token started can be cancelled, and an unknown id is reported as not found. Read `accessInfo` for the outcome.

Firecrawl Usage

Check the Firecrawl account's remaining credits and tokens plus its current queue depth and concurrency limit. Call it before a large crawl, batch scrape or extraction so an exhausted balance surfaces as a clear answer rather than a job that dies halfway, and after one to see what it cost. The figures are account-wide, so a token not permitted to see cost gets the connection's health without the numbers — read `accessInfo` rather than reading a missing figure as zero.

Read Crawl Result Page

Read one bounded page of an owned crawl or batch scrape. Pending/failed jobs are not completed empty results. sourceReadInfo carries read state, coverage and nextArgs; use the same tool for continuation. Current egress and field grants apply on every page.

Acquire Decision Crawl

Hosted decision runs only: acquire a bounded fresh crawl or resume this worker token's jobId, then read one result window. Reuses the recorded launch for the same run; uncertain launches are never automatically repeated. The decision runtime polls pending jobs within its deadline. Source acquisition spends Firecrawl credits separately from model usage. sourceReadInfo.nextArgs resumes an owned job; source limits do not establish whole-site coverage.