Migrate from Firecrawl in One Afternoon

Goal: Move a working Firecrawl integration to fastCRW in a single afternoon. This guide covers the four mechanical changes (base URL, SDK package, env var, extraction shape) plus the behavioral differences you must validate before going to production.

Time estimate: 15 minutes of code changes, 30–60 minutes of validation.

What transfers without changes

The core request shape is intentionally compatible. These work against fastCRW with no modification when you point the client at the right base URL:

• /v1/scrape — same body fields (url, formats, onlyMainContent, waitFor, etc.)
• /v1/crawl + /v1/crawl/{id} polling
• /v1/map
• /v1/search
• Authorization: Bearer <key> header

Step 1 — Swap the base URL

Every engine call goes to https://api.fastcrw.com instead of https://api.firecrawl.dev.

curl -X POST https://api.firecrawl.dev/v1/scrape \
  -H "Authorization: Bearer $FIRECRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://news.ycombinator.com",
    "formats": ["markdown"]
  }'

curl -X POST https://api.fastcrw.com/v1/scrape \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://news.ycombinator.com",
    "formats": ["markdown"]
  }'

Self-hosted: if you run your own crw-server, replace https://api.fastcrw.com with your server's address (e.g. http://crw:3000). Authentication is optional on self-hosted; disable it with [server] auth_required = false in your config.toml.

:::note{title="Keeping the Firecrawl SDK? Just repoint the API URL"} You don't have to swap SDKs. fastCRW mirrors Firecrawl's v2 API under a dedicated /firecrawl/* namespace, so an existing Firecrawl client works once you point its base URL at fastCRW. How you set it differs by language because the two SDKs build request URLs differently:

TypeScript / JavaScript (@mendable/firecrawl-js) — include /firecrawl in apiUrl. The SDK concatenates the path, so requests land on /firecrawl/v2/*:

const app = new FirecrawlApp({
  apiKey: process.env.CRW_API_KEY,
  apiUrl: "https://api.fastcrw.com/firecrawl",
});

Python (firecrawl-py) — use the bare host, no /firecrawl. The Python SDK resolves endpoint paths against the host root (via urljoin), so a /firecrawl suffix is silently dropped; you land on the equivalent root /v2/* surface, which is the same engine:

app = FirecrawlApp(
    api_key=os.environ["CRW_API_KEY"],
    api_url="https://api.fastcrw.com",  # resolves to /v2/* — the same compat surface
)

Both reach the same Firecrawl-compatible engine. The v2 SDK is the supported drop-in; the legacy v1 SDK is compatible only for scrape. :::

Step 2 — Swap the SDK package

Python

pip install firecrawl-py

from firecrawl import FirecrawlApp

app = FirecrawlApp(api_key="fc-...")

result = app.scrape_url(
    "https://news.ycombinator.com",
    params={"formats": ["markdown"]},
)
print(result["markdown"][:500])

pip install crw

from crw import CrwClient

client = CrwClient()  # reads CRW_API_KEY from env

result = client.scrape(
    "https://news.ycombinator.com",
    formats=["markdown"],
)
print(result["markdown"][:500])

TypeScript / JavaScript

npm install @mendable/firecrawl-js

import FirecrawlApp from "@mendable/firecrawl-js";

const app = new FirecrawlApp({ apiKey: "fc-..." });

const result = await app.scrapeUrl("https://news.ycombinator.com", {
  formats: ["markdown"],
});
console.log(result.markdown?.slice(0, 500));

npm install crw-sdk

import { CrwClient } from "crw-sdk";

const client = new CrwClient({ apiKey: process.env.CRW_API_KEY });

const result = await client.scrape("https://news.ycombinator.com", {
  formats: ["markdown"],
});
console.log(result.markdown?.slice(0, 500));

Step 3 — Rename the environment variable

Shell (.env or export):

# remove
# export FIRECRAWL_API_KEY=fc-...

# add
export CRW_API_KEY=crw-...

Both SDKs read the variable automatically. No code change is needed once the env var is in place.

Step 4 — Port LLM extraction

Firecrawl exposes a standalone /v1/extract route. fastCRW supports both styles: single-URL extraction runs through /v1/scrape with formats: ["json"] and a jsonSchema (response field data.json), and multi-URL extraction has a native async POST /v1/extract (URLs + prompt/schema) that returns a per-URL results array via GET /v1/extract/{id}.

Multi-URL extraction: send { "urls": [...], "prompt"/"schema": ... } to /v1/extract, then poll /v1/extract/{id} for results (one { url, status, data, error } per URL).

from firecrawl import FirecrawlApp

app = FirecrawlApp(api_key="fc-...")

schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "price": {"type": "string"},
    },
    "required": ["title", "price"],
}

result = app.extract(
    ["https://example.com/product"],
    {"schema": schema},
)
print(result)

from crw import CrwClient

client = CrwClient()  # reads CRW_API_KEY from env

schema = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "price": {"type": "string"},
    },
    "required": ["title", "price"],
}

# Pass json_schema — the SDK adds "json" to formats automatically
result = client.scrape(
    "https://example.com/product",
    json_schema=schema,
)
print(result["json"])

import FirecrawlApp from "@mendable/firecrawl-js";

const app = new FirecrawlApp({ apiKey: "fc-..." });

const schema = {
  type: "object",
  properties: {
    title: { type: "string" },
    price: { type: "string" },
  },
  required: ["title", "price"],
};

const result = await app.extract(["https://example.com/product"], {
  schema,
});
console.log(result.data);

import { CrwClient } from "crw-sdk";

const client = new CrwClient({ apiKey: process.env.CRW_API_KEY });

const schema = {
  type: "object",
  properties: {
    title: { type: "string" },
    price: { type: "string" },
  },
  required: ["title", "price"],
};

// Pass jsonSchema — the SDK adds "json" to formats automatically
const result = await client.scrape("https://example.com/product", {
  jsonSchema: schema,
});
console.log(result.json);

Behavioral differences to validate

The following table lists the gaps documented in COMPATIBILITY-firecrawl.md. Validate each that applies to your workload before switching production traffic.

5-minute verification checklist

Run these five commands against a real URL from your workload after making the changes above. Use a URL you can manually inspect in your browser.

1. Smoke test — basic scrape

curl -s -X POST https://api.fastcrw.com/v1/scrape \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://news.ycombinator.com", "formats": ["markdown"]}' \
  | python3 -m json.tool | head -30

Expected: "success": true, "markdown" field with real content.

2. Crawl smoke test

curl -s -X POST https://api.fastcrw.com/v1/crawl \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "maxDepth": 1, "maxPages": 3}' \
  | python3 -m json.tool

Expected: {"success": true, "id": "<job-id>"}. Poll GET /v1/crawl/{id} until "status": "completed".

3. Map smoke test

curl -s -X POST https://api.fastcrw.com/v1/map \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "maxDepth": 1}' \
  | python3 -m json.tool | head -20

Expected: "links": [...] with discovered URLs.

4. Search smoke test

curl -s -X POST https://api.fastcrw.com/v1/search \
  -H "Authorization: Bearer $CRW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "fastcrw web scraper", "limit": 3}' \
  | python3 -m json.tool | head -30

Expected: "data": [...] with search results. (Against a self-hosted engine the same call returns "data": {"results": [...]} — see Response shapes.)

5. SDK round-trip (Python)

import os
from crw import CrwClient

client = CrwClient()  # CRW_API_KEY must be set

# Basic scrape
result = client.scrape("https://example.com", formats=["markdown"])
assert result.get("markdown"), "markdown field is empty"
print("scrape: OK")

# Map
links = client.map("https://example.com", max_depth=1)
assert isinstance(links, list) and len(links) > 0, "map returned no links"
print(f"map: OK ({len(links)} links)")

print("All checks passed — safe to promote to production.")

Feature-detect the engine

Use GET /v1/capabilities to confirm what the target engine supports before sending calls that depend on optional features (LLM extraction, file parsing, search):

curl -s https://api.fastcrw.com/v1/capabilities \
  -H "Authorization: Bearer $CRW_API_KEY" \
  | python3 -m json.tool

The response lists llm, search, documents.parsers, and formats so your code can branch instead of assuming.

What is not in scope

The following Firecrawl Cloud capabilities have no equivalent in fastCRW and are not planned:

• /v1/deep-research (Spark model pipeline)
• /v1/agent (AI agent sessions)
• Fire-engine proprietary anti-bot layer

Bring-your-own-proxy IS supported on a self-hosted instance: set a pool in [proxy], or pass proxy on the scrape body. See Proxies.

For any capability not in the matrix above, check COMPATIBILITY-firecrawl.md which is the authoritative reference.