Introducing Web Scrape

Web Scrape extracts the main content of a URL as Markdown for 1 credit per completed request.

What you get

• markdown: the page’s main content with its headings, paragraphs, lists, tables, links and images, with the site’s header, footer and navigation removed.
• title, description, author, publishedAt, siteName, language and image whenever the page declares them.
• finalUrl and statusCode: the address after redirects and the status the site answered with.
• images: every picture in markdown, in the order it appears, with its alt text.

How it works

Send a public http or https URL of up to 2,048 characters. Web Scrape opens the page, renders pages that a script draws, such as shop product pages, and keeps the content a reader comes for: the text, the tables and the pictures. The result is clean Markdown, ready to store, search or analyze.

maxChars sets the length, from 1,000 to 200,000 characters (100,000 by default). A long page stops at a paragraph break and sets truncated: true. country (US by default) sets the language and region the page is requested for. A PDF, a page that answers 404 or 410, or a site that refuses the request ends the task with PAGE_UNAVAILABLE and the site’s answer in the message.

Where it helps

• Read the pages AI answers cite, to see what content earns a citation.
• Pass a link from GOOGLE_SERP results to read the pages that rank for a query.
• Follow competitors’ product and pricing pages as text you compare over time.
• Read a page from your AI assistant through the MCP tool scrape_page.

Send a request

Send the request with your API key in the Authorization header.

curl -X POST https://api.querying.ai/v1/async/task \
  -H "Authorization: Bearer $QUERYING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "taskType": "WEB_SCRAPE",
    "payload": {
      "url": "https://docs.querying.ai/quickstart",
      "country": "US"
    }
  }'

Request fields

taskTypestringRequired

The engine to run, such as CHATGPT or GOOGLE.

payload.urlstringRequired

The page to read: a public http or https address, up to 2,048 characters.

payload.maxCharsinteger

Length cap for markdown, 1,000 to 200,000. Default 100,000. A longer page ends at a paragraph break and carries truncated.

payload.countrystring

Country the page is requested from, which sets its language and region. Default US.

webhook.urlstring

An HTTPS address on your server. The finished task is posted there the moment it completes.

idempotencyKeystring

Your own ID for the request. Each key creates one task per account, so a retried send stays a single task.

Response

A finished task looks like this, trimmed for length.

{
  "success": true,
  "task": {
    "id": "910017b1-4fd5-4bc7-8895-bbe6b1b0d1df",
    "taskType": "WEB_SCRAPE",
    "status": "COMPLETED",
    "createdAt": "2026-10-02T15:48:34.088Z"
  },
  "credits": { "creditsToCharge": 1, "creditsCharged": 1 },
  "response": {
    "url": "https://docs.querying.ai/quickstart",
    "finalUrl": "https://docs.querying.ai/quickstart",
    "statusCode": 200,
    "title": "AI Search Data API Quickstart",
    "description": "Submit a task, receive a result — with webhooks or by polling.",
    "siteName": "querying.ai",
    "language": "en",
    "markdown": "# AI Search Data API Quickstart\n\n[한국어 시작 가이드](https://querying.ai/ko/guides/quickstart) · [API plans and credits](https://querying.ai/en/pricing)\n\n## 1\\. Set your credentials\n\n```shellscript\nexport BASE=\"https://api.querying.ai\"\nexport API_KEY=\"<your-key>\"\n```\n\n…",
    "images": [],
    "truncated": false
  }
}

task.statusstring

QUEUED, PROCESSING, then COMPLETED with the answer or FAILED with error.

credits.creditsChargedinteger

Credits charged for the finished task. A failed task charges 0.

response.markdownstring

The page’s main content as Markdown: headings, paragraphs, lists, tables, links and images.

response.titlestring

The page title, with description, author, siteName and language when the page declares them.

response.publishedAtstring

The publish date the page declares, in ISO 8601.

response.finalUrlstring

The address after redirects, with statusCode.

response.imagesarray

The pictures in markdown, in reading order, each with url and alt.

response.truncatedboolean

true when markdown reached maxChars and ends at a paragraph break.

Receive the result

The request returns the task right away with status QUEUED. Poll GET /v1/async/task/{id} until status is COMPLETED or FAILED, or add webhook.url and the finished task arrives at your server with the same task, credits and response. Results stay available for 24 hours after a task finishes.

curl https://api.querying.ai/v1/async/task/$TASK_ID \
  -H "Authorization: Bearer $QUERYING_API_KEY"

Pricing

Each completed page uses 1 credit.