Docs atdocs.fastcrw.com. Hosted API athttps://api.fastcrw.com. Self-hosted athttp://localhost:3000. Five endpoints:/v1/search,/v1/scrape,/v1/crawl,/v1/map,/v1/extract.
crw, TypeScript crw-sdkTyped clients for every endpoint. Use them instead of raw HTTP when a task asks for the SDK. Full reference: /sdk-reference.
# pip install crw
from crw import CrwClient
client = CrwClient() # reads CRW_API_KEY; defaults to https://api.fastcrw.com
page = client.scrape("https://example.com", formats=["markdown", "links"])
# page is a plain dict (no .data wrapper)
print(page["markdown"]) # str: clean page markdown, links kept inline as [text](url)
print(page["links"]) # list[str]: every absolute URL on the page, no anchor text
print(page["metadata"]["title"]) # str
# Run with: python3 scrape.py
Signature: scrape(url, formats=None, only_main_content=None, include_tags=None,
exclude_tags=None, *, render_js=None, renderer=None, wait_for=None, json_schema=None,
**kwargs) -> dict. Extra keyword arguments such as css_selector
are passed to the API as they are.
css_selector keeps only the matching elements, so the markdown is just
their links as [title](url), in page order. No exploratory scrape needed.
import json, re
from crw import CrwClient
page = CrwClient().scrape("https://news.ycombinator.com", formats=["markdown"], css_selector=".titleline > a")
pairs = re.findall(r"\[([^\]]+)\]\((https?://[^)\s]+)\)", page["markdown"]) # [(title, url), ...]
print(json.dumps([{"rank": i, "title": t, "url": u} for i, (t, u) in enumerate(pairs[:5], 1)], indent=2))
// npm install crw-sdk
import { CrwClient } from "crw-sdk";
const crw = new CrwClient(); // reads CRW_API_KEY
const page = await crw.scrape("https://example.com", { formats: ["markdown"] });
console.log(page.markdown);
Other methods: crawl(url), map(url),
search(query, limit=5), extract(urls, schema={...}).
scrape(url, json_schema={...}) runs managed LLM extraction:
on the hosted FREE plan it returns an error, so parse
markdown or links there instead.
Auth (hosted only):
Authorization: Bearer $CRW_API_KEY.
Self-hosted (http://localhost:3000) needs no header.
Body:
{ "query": string, "limit": number (default 5, max 20), "lang"?: "en", "tbs"?: "qdr:d|w|m|y", "sources"?: ["web"|"news"|"images"] }
#!/usr/bin/env python3
"""Search CRW and print top results as JSON."""
import json, os, sys, urllib.request
API_KEY = os.environ.get("CRW_API_KEY")
# Default to the hosted API. Self-hosting? Set CRW_API_URL=http://localhost:3000
BASE = os.environ.get("CRW_API_URL", "https://api.fastcrw.com")
URL = f"{BASE}/v1/search"
headers = {"Content-Type": "application/json"}
if API_KEY:
headers["Authorization"] = f"Bearer {API_KEY}"
elif BASE.startswith("https://api.fastcrw.com"):
sys.exit("error: CRW_API_KEY is required for the hosted API. "
"Get one at https://fastcrw.com (1000 free credits) "
"or set CRW_API_URL=http://localhost:3000 to self-host.")
req = urllib.request.Request(
URL,
data=json.dumps({"query": "renewable energy trends 2024", "limit": 3}).encode(),
headers=headers,
method="POST",
)
with urllib.request.urlopen(req) as r:
body = json.loads(r.read()) # body == {"success": true, "data": [...]}
# Results live under body["data"]. Each row has `title`, `url`, `snippet`,
# `description`, `position`, `score`, `category` — print the LLM-ready subset.
print(json.dumps(
[{"title": x["title"], "url": x["url"], "snippet": x["snippet"]}
for x in body["data"]],
indent=2,
))
# Run with: python3 search.py (file is python3-only — no `python` alias needed)
curl -X POST https://api.fastcrw.com/v1/search \
-H "Authorization: Bearer $CRW_API_KEY" \
-H "Content-Type: application/json" \
-d '{"query":"renewable energy trends 2024","limit":3}'
The crw CLI projects directly to the title/url/snippet
schema with --json --fields — no jq reshape
needed:
# Install: brew install fastcrw/crw/crw (or: cargo install crw-cli)
crw search "renewable energy trends 2024" --json --fields title,url,snippet --limit 3
# Available fields: title, url, description, snippet, position, score, category
# --json is shorthand for --format json (also accepts --format text|markdown)
# Self-hosted: no auth. Hosted: export CRW_API_KEY=crw_live_...
{
"success": true,
"data": [
{"url": "https://...", "title": "...",
"description": "...", "snippet": "...",
"position": 1, "score": 9.5}
]
}
POST /v1/scrape {"url":"...","formats":["markdown"]} — one URL → markdownPOST /v1/map {"url":"...","limit":50} — discover URLsPOST /v1/crawl {"url":"...","maxPages":5} — async multi-page; poll GET /v1/crawl/{id}POST /v1/scrape with formats:["json"] + jsonSchema: structured extract
pip install crw ·
npx crw-mcp ·
brew install fastcrw/crw/crw ·
curl -fsSL https://fastcrw.com/install | sh ·
docker run -p 3000:3000 ghcr.io/fastcrw/crw:latest