How to collect thousands of AI answers a day: batches, concurrency and webhooks

Send a daily job of thousands of ChatGPT, Gemini and Perplexity prompts as one batch request, size it to your plan's concurrency and credits, retry without duplicates and store every answer within its 24-hour window.

Pick the endpoint by the size of the job

Three endpoints create many tasks in one request. Each creates ordinary tasks, so polling, webhooks, credits and plan limits work the same as for a single task. The difference is how many tasks they take and who handles tasks over your plan's limit. Full reference: Batches and grids.

• Up to 500 tasks, with your own retry loop: POST /v1/async/task/batch. The response returns every task id at once.
• A daily job of thousands, or more tasks than your concurrency: POST /v1/batches. The server paces the items, so your code sends one request and listens.
• One prompt asked from many points around a place: POST /v1/async/task/grid, up to a 9 × 9 grid.

Submit a daily job as one batch

A batch carries an items array, one webhook.url for every item, optional metadata and an idempotencyKey for the batch itself. Each item is a normal task request with an optional customId that comes back with its result. Date the batch key, and sending the same day's batch twice returns the first one.

curl -X POST https://api.querying.ai/v1/batches \
  -H "Authorization: Bearer $QUERYING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "idempotencyKey": "daily-2026-11-05",
    "webhook": { "url": "https://hooks.example.com/querying" },
    "metadata": { "run": "daily" },
    "items": [
      { "customId": "p0001-chatgpt", "idempotencyKey": "2026-11-05:p0001:chatgpt", "taskType": "CHATGPT", "payload": { "prompt": "best CRM for a small team", "country": "US" } },
      { "customId": "p0001-gemini", "idempotencyKey": "2026-11-05:p0001:gemini", "taskType": "GEMINI", "payload": { "prompt": "best CRM for a small team", "country": "US" } }
    ]
  }'

The response is 202 with the batch id, its status and counts by item status. Every item is validated before anything is stored: one invalid item returns 400 VALIDATION_ERROR with the failing indexes, so a malformed prompt list fails fast and costs nothing.

What sets your throughput

Two numbers set how much a plan collects. Concurrency is how many tasks you have queued or processing at once, and the server releases batch items into that room as earlier tasks finish. Credits set how many answers the month holds. The table divides each plan's monthly credits by 30.

The free plan starts with 500 credits and 2 concurrent tasks, enough to test a batch end to end. The edge allows 1,200 requests per minute per API key, and a 500 item array counts as one request, so batching also keeps a large job well inside the request rate. Limits and codes are in Credits and limits.

curl https://api.querying.ai/v1/async/status \
  -H "Authorization: Bearer $QUERYING_API_KEY"

{ "queuedTasks": 19, "processingTasks": 6, "concurrency": { "used": 25, "max": 200 } }

GET /v1/async/status is free to call. concurrency.max - concurrency.used is the room left for new tasks, which tells a scheduler how fast to send single tasks alongside a running batch.

A worked daily job: 1,000 prompts on four engines

Take 1,000 prompts asked once a day in one country on ChatGPT, Perplexity, Gemini and Google AI Overviews. That is 4,000 tasks, well under the 10,000 item limit of one batch. The four engines cost 4.5 credits per prompt, so the job uses 4,500 credits a day and 135,000 a month. Pro at $200 a month holds 170,000 credits and 200 concurrent tasks.

• Build the items from your prompt table: one item per prompt and engine, with customId set to your own prompt id and engine.
• Key each item by date, prompt and engine, for example 2026-11-05:p0001:chatgpt.
• Send one POST /v1/batches with a dated batch key and your webhook URL.
• Write each task webhook to your store, and close the day's run on batch.completed.
• Sweep GET /v1/batches/{id}/items?status=failed and decide which prompts to send again tomorrow.

To grow the job, add prompts before you add repeats. Monthly cost scales with prompts × engines × countries × runs, worked through in How much does AI visibility tracking cost?.

Retry without duplicates

• Give every task an idempotencyKey. A repeat of the same key returns the existing task, so a retry after a timeout or a crash creates no second answer and no second charge. Keys stay reserved for at least 30 days.
• On 429 and 503 from the submit endpoints, wait the seconds in Retry-After and resubmit with the same key.
• Edge 429 responses carry no JSON body. Back off with an exponential delay.
• Inside POST /v1/batches, items refused for capacity stay PENDING and the server tries them again. INSUFFICIENT_CREDITS rejects the item, so top up credits before a large run.

Store answers before they expire

A finished task delivers a webhook with its result, and a batch posts one batch.completed event when every item has finished. Delivery is at least once: deduplicate on the querying-webhook-id header and make the write idempotent. Signatures and retry timing are in Webhooks.

curl https://api.querying.ai/v1/batches/3f6b2a10-... \
  -H "Authorization: Bearer $QUERYING_API_KEY"

curl "https://api.querying.ai/v1/batches/3f6b2a10-.../items?status=completed&include=response&limit=100" \
  -H "Authorization: Bearer $QUERYING_API_KEY"

Polling is the safety net. GET /v1/batches/{id} returns counts by item status, and the items endpoint pages through results 100 at a time with include=response. Result bodies stay readable for 24 hours after each task completes, and the batch itself stays readable for 30 days, so save answers as they arrive and run the sweep within the day.