API overview

The HTTP API runs AzScraper: organizations hold projects, you enqueue crawl jobs (search or product-details), a worker consumes work from a queue (Redis / BullMQ), and results are stored for listing and export (CSV or JSON).

What you can do

Area	Behavior
Projects	Group crawl jobs. List, create, and update (name, archived) per organization.
Jobs	Create jobs with type search (keywords) or details (ASINs/URLs), targets, optional config (e.g. lang). Poll job status, list by project, retry or cancel.
Results	Fetch result rows as JSON after the job completes.
Export	Download the same data as format=json or format=csv.
Schedules	Optional cron-based schedules that enqueue crawls for a project.
Auth	JWT issued by POST /auth/login or POST /auth/register; send Authorization: Bearer <accessToken> on protected routes.

Architecture (high level)

Client ──HTTPS──▶ NestJS API ──▶ MongoDB (jobs, projects, users, …)
                      │
                      └──▶ Redis queue ──▶ Worker (crawl execution)

• API (apps/api): REST JSON, Swagger UI, global error envelope.
• Worker: Processes queued crawl jobs; updates job status and result sets.

Interactive reference

When you run the API locally or self-host it, open /api/docs on the same origin as the API (for example http://localhost:3001/api/docs) for OpenAPI/Swagger: request schemas, try-it-out, and the authoritative route list. Production base URL is environment-specific; placeholders below use https://api.azscraper.com and http://localhost:3001.

Related pages

• Get started — base URL, env vars, first-request flow.
• Authentication — register, login, Bearer tokens, handoff.
• Get data — projects, jobs, results, export, schedules, quick reference table.
• Webhooks — delivery notifications (product status).
• Error taxonomy — error codes and HTTP status mapping.
• Sample responses — example JSON payloads.
• Rate limits & concurrency — credits, queue, throttling.

Referral and admin referral endpoints are summarized in Referral API.