Web Data · DomScan product

Web Scraping API

Fetch public web pages and documents as HTML, Markdown, LLM-ready text or plain text, one page at a time, in batches, or as a whole-site crawl.

What you get

Fetch public web pages and documents as HTML, Markdown, LLM-ready text or plain text, one page at a time, in batches, or as a whole-site crawl.

SynchronousAsynchronousRESTMCPSDK

Choose this product when

Who it is for

Use focused APIs when a product or workflow needs current public page evidence in a structured response.

Choose this product when

Choose this when you need clean HTML, text or Markdown from public pages, PDFs or spreadsheets, from one page up to a whole site, with rendering or asynchronous collection only when supported and necessary.

What you get

Fetch public web pages and documents as HTML, Markdown, LLM-ready text or plain text, one page at a time, in batches, or as a whole-site crawl.

Endpoints

See what each endpoint accepts and returns, its modes and its limits before you integrate.

POST /v1/scrape View API reference
POST /v1/scrape/jobs View API reference
GET /v1/scrape/jobs/:job_id View API reference
GET /v1/scrape/jobs/:job_id/results View API reference
POST /v1/crawl/jobs View API reference
GET /v1/crawl/jobs/:job_id View API reference
GET /v1/crawl/jobs/:job_id/results View API reference

Credits and authentication

0-20 Authentication required. See the API reference for the exact cost, including any per-item pricing.

Example operation

Use documented operations to collect, assess, enrich, verify, compare, protect, or monitor public evidence.

POST /v1/scrape Credits: 1-20/item

Body parameters

Parameter Type required
url string required
mode string
Allowed values standardresilientrenderedrendered_resilient
Default standard
optional
output string
Allowed values htmltextmarkdownllm
Default html
optional
only_main_content boolean
Default false
optional
include_selectors string[] optional
exclude_selectors string[] optional
links boolean
Default false
optional
headers object optional
cookies object optional

Response fields

Field Type
data object
data.url string
data.final_url string
data.status integer | null
data.outcome string
data.content_type string | null
data.content string
data.bytes integer
data.truncated boolean
data.redirect_count integer
data.attempt_count integer
data.duration_ms integer
data.fetched_at string
data.headers object
data.rendered boolean
data.links[] object[]
data.links[] object
data.links[].url string
data.links[].text string
data.document object
data.document.kind string
data.document.title string | null
data.document.pages integer | null
data.document.sheets[] string[]
data.document.truncated boolean
data.document.error string
data.depth integer
data.forwarded object
data.forwarded.header_names[] string[]
data.forwarded.cookie_names[] string[]
data.forwarded.scope string
billing object
billing.credits_charged integer
billing.credits_refunded integer
billing.credits_per_item integer
billing.credits_net integer
billing.policy string

Example request

curl -X POST "https://domscan.net/v1/scrape" \
  -H "Content-Type: application/json" \
  -H "X-API-Key: $DOMSCAN_API_KEY" \
  -d '{
  "url": "https://example.com/",
  "mode": "standard",
  "output": "markdown"
}'
const response = await fetch("https://domscan.net/v1/scrape", {
  method: "POST",
  headers: { "X-API-Key": process.env.DOMSCAN_API_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({
    "url": "https://example.com/",
    "mode": "standard",
    "output": "markdown"
  }),
});
const data = await response.json();
console.log(data);
import os
import requests

response = requests.post(
    "https://domscan.net/v1/scrape",
    headers={"X-API-Key": os.environ["DOMSCAN_API_KEY"]},
    json={
        "url": "https://example.com/",
        "mode": "standard",
        "output": "markdown",
    },
)
print(response.json())
package main

import (
	"fmt"
	"io"
	"net/http"
	"os"
	"strings"
)

func main() {
	body := strings.NewReader(`{
	  "url": "https://example.com/",
	  "mode": "standard",
	  "output": "markdown"
	}`)
	req, err := http.NewRequest(http.MethodPost, "https://domscan.net/v1/scrape", body)
	if err != nil {
		panic(err)
	}
	req.Header.Set("X-API-Key", os.Getenv("DOMSCAN_API_KEY"))
	req.Header.Set("Content-Type", "application/json")

	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		panic(err)
	}
	defer resp.Body.Close()

	data, err := io.ReadAll(resp.Body)
	if err != nil {
		panic(err)
	}
	fmt.Println(resp.Status, string(data))
}
require "json"
require "net/http"

uri = URI("https://domscan.net/v1/scrape")
request = Net::HTTP::Post.new(uri)
request["X-API-Key"] = ENV.fetch("DOMSCAN_API_KEY")
request["Content-Type"] = "application/json"
request.body = JSON.generate({
  "url" => "https://example.com/",
  "mode" => "standard",
  "output" => "markdown"
})

response = Net::HTTP.start(uri.hostname, uri.port, use_ssl: true) do |http|
  http.request(request)
end
puts JSON.pretty_generate(JSON.parse(response.body))

Example response

{
  "data": {
    "url": "https://example.com",
    "final_url": "https://example.com",
    "status": 1,
    "outcome": "success",
    "content_type": "domain",
    "content": "example",
    "bytes": 1,
    "truncated": false,
    "redirect_count": 1,
    "attempt_count": 1,
    "duration_ms": 1,
    "fetched_at": "2026-08-27T12:00:00Z",
    "headers": {},
    "rendered": false,
    "links": [
      {
        "url": "https://example.com",
        "text": "example"
      }
    ],
    "document": {
      "kind": "pdf",
      "title": "example",
      "pages": 1,
      "sheets": [
        "example"
      ],
      "truncated": false,
      "error": "DOCUMENT_UNREADABLE"
    },
    "depth": 1,
    "forwarded": {
      "header_names": [
        "example"
      ],
      "cookie_names": [
        "example"
      ],
      "scope": "requested_origin"
    }
  },
  "billing": {
    "credits_charged": 1,
    "credits_refunded": 1,
    "credits_per_item": 1,
    "credits_net": 1,
    "policy": "effort_based"
  }
}

HTTP status codes

  • 200 OK
  • 400 Bad Request
  • 401 Unauthorized
  • 402 Payment Required
  • 403 Forbidden
  • 409 HTTP 409
  • 422 HTTP 422
  • 429 Too Many Requests
  • 503 Service Unavailable

Frequently asked questions

Where can I find the Web Scraping request and response reference?

Use the API reference for Web Scraping to review parameters, response fields, examples, status codes, and supported operation modes.

How should I interpret unknown or incomplete results from Web Scraping?

Unknown or partial means we could not fully confirm the result. Keep the status, confidence, freshness and any limits in mind when you act on it.

How is Web Scraping priced?

The price depends on the check and the mode you choose. The API reference shows the current credit cost, whether you need an API key, and any per-item pricing.

Used by people at these companies

InstantOutseerMongoDBRespondentSage Expense ManagementInstantlyD.R. HortonWhatConvertsAdobeLLM Pulse