Skip to content

Webhooks

Webhooks send signed HTTP POST requests to your endpoint when crawl or monitoring events occur. Configure them with a project API key. All paths below include the /v1/key prefix.

Supported events

Event When it is sent
job.completed A crawl job completes.
job.failed A crawl job fails.
schedule.triggered Accepted as a configuration and test event, but scheduled crawls do not currently emit this event.
product.changed A monitored product change is detected.

List current configuration and delivery log

GET /v1/key/projects/{projectId}/webhooks

curl -sS "$API_BASE/v1/key/projects/$PROJECT_ID/webhooks" \
  -H "Authorization: Bearer $API_KEY"

The response includes data.configs and recent data.logs. Newly configured endpoints have status: "need-verify"; existing endpoints must also be verified before event delivery. Keep every returned secret confidential.

Replace webhook configuration

PATCH /v1/key/projects/{projectId}/webhooks

This replaces all webhook configurations for the project. You can send up to 20 configurations. Omit id to create a config; omit secret or provide an empty string to generate a signing secret. Save the returned secret securely.

curl -sS -X PATCH "$API_BASE/v1/key/projects/$PROJECT_ID/webhooks" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"configs":[{"name":"crawl-events","url":"https://example.com/hooks/azscraper","triggerAction":"job.completed","enabled":true}]}'

Each configuration needs a destination url and a triggerAction from the supported event list. enabled defaults to true. Saving a new endpoint or changing its URL or secret puts it in need-verify. Only endpoints with status: "active" receive real events.

Send a test event

POST /v1/key/projects/{projectId}/webhooks/test

curl -sS -X POST "$API_BASE/v1/key/projects/$PROJECT_ID/webhooks/test" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"webhookId":"<webhook-id>"}'

For a config in need-verify, this queues a signed fake webhook.verify event. The endpoint must validate the signature and return HTTP 200; invalid signatures should return HTTP 401. A verification request times out after 5 seconds. A queued verification becomes active only after HTTP 200. Use the dashboard's Verify button after setup, or poll the config and delivery log when using the API. For an already active config, this sends a normal sample event; optionally provide triggerAction to test a specific supported event.

Verify a delivery

The JSON body has this shape:

{
  "id": "<delivery-id>",
  "event": "job.completed",
  "createdAt": "<ISO-8601 timestamp>",
  "projectId": "<project-id>",
  "data": {}
}

AzScraper sends these headers:

Header Value
X-AzScraper-Signature t=<unix-seconds>,v1=<hex-hmac>
X-AzScraper-Event Event name, such as job.completed.
X-AzScraper-Delivery Stable delivery ID; use it to deduplicate retries.
X-AzScraper-Webhook-Id Configured webhook ID.

For webhook.verify, the payload contains data.verification: true. Return HTTP 200 only after the signature is valid and HTTP 401 if it is invalid. Redirect responses are not followed.

Verify the HMAC-SHA256 signature with the webhook's secret and the raw request body. The signed message is <timestamp>.<raw-body>; compare the computed signature with v1 using a constant-time comparison and reject timestamps outside your replay window. The default verification tolerance in the shared verifier is five minutes.

Failed deliveries are retried up to five attempts with exponential backoff. Use the delivery ID to make your handler idempotent.