Skip to content

LLM-ready export (format=llm)

Use format=llm when feeding crawl results into LLMs, RAG pipelines, or any downstream system that needs a normalized, deduped, versioned JSON contract instead of raw parser output.

Request

GET /private/jobs/{jobId}/export?format=llm
Authorization: Bearer <accessToken>

Requires a completed job and the same project access as other export formats.

Response

  • Content-Type: application/json
  • Attachment: job-{jobId}-results.llm.json
curl -sS "$API_BASE/private/jobs/JOB_ID/export?format=llm" \
  -H "Authorization: Bearer $TOKEN" \
  -o job-results.llm.json

Envelope shape

Field Meaning
schemaVersion Contract version (currently 1.0.0) — pin on major when integrating
kind "search" or "details"
generatedAt Job completion time (ISO 8601) or null
marketplace Marketplace code (e.g. US) or null
language Crawl language or null
productCount Number of products after dedupe
products Array of normalized products

Every product includes the same keys (asin, title, brand, url, imageUrl, price, currency, rating, reviewCount, features, categories, keyword). Values that do not apply to the job type are explicit null or empty arrays — fields are never omitted.

Why use llm vs json?

format=json format=llm
Shape Raw stored crawl items Normalized profile
Dedupe No Yes (order-preserving)
Version field No schemaVersion
Key stability Parser-dependent Fixed key order

Schema & versioning

Machine-readable JSON Schema (for validation in your pipeline):

packages/crawler-core/schemas/llm-ready-output.schema.json

Breaking changes bump the major schemaVersion. Maintainer reference: repo docs/llm-ready-export.md.

See also