Structured extraction with LLM

Short answer: Structured extraction with an LLM is the technique of giving the model a JSON Schema and forcing its output to validate against that schema, so the result is machine-readable on the first try. Combined with a web scraper, it turns any page — contact page, product card, job listing — into a typed JSON object via one HTTP call.

How it works

The server fetches and renders the page, converts the rendered DOM to markdown or plain text, and hands that text plus your JSON Schema to an LLM. The model is constrained by one of three mechanisms: OpenAI-style tool calling with the schema as the tool definition, JSON-mode with grammar-guided sampling, or post-validation with retry. The schema acts as both prompt scaffolding and output contract; the caller never has to write a regex, an XPath, or a BeautifulSoup selector.

When to use it

Structured extraction is the right tool whenever the target page is unstable, varies in layout across the site, or carries information you want typed rather than free-form. Concrete use cases:

• Contact pages — pull {email, phone, address} regardless of markup.
• Product cards — pull {title, price, currency, in_stock, sku} across thousands of listings.
• Job listings — pull {title, company, location, salary_range, remote} from ATS pages with no shared template.
• Pricing pages — pull a list of plan objects when each vendor's HTML is different.

When the page is rigid and well-templated, a selector-based scrape is faster and cheaper. When you don't know the layout in advance, LLM extraction wins.

How fastCRW does it

POST a JSON body with url, a schema (any valid JSON Schema), and an optional prompt for natural-language hints to /v1/extract. Both fields can coexist — the prompt steers and the schema constrains. BYOK is supported via llmApiKey, llmProvider, and llmModel, so your billing relationship is with the LLM vendor directly. See the extract guide for end-to-end examples.

Frequently asked

Is the schema or the prompt more important?

The schema. The prompt nudges the model toward the right fields; the schema constrains the output shape. A schema with no prompt usually works; a prompt with no schema returns free-form text.

What JSON Schema features are supported?

Object, array, string, number, boolean, enum, and nested types. Required fields and descriptions are passed through to the model and improve accuracy.

How is this different from regular scraping?

Regular scraping returns raw markdown or HTML and leaves parsing to you. Structured extraction returns typed JSON validated against your schema, so the parsing step is server-side.

← CRW docs home