Scrape API (preview)

bytetourist’s proxy layer gives you the egress. The Scrape API is the planned layer on top: turn a URL into clean, LLM-ready data — running on the IPs you already own, instead of paying a scraping vendor that rents theirs.

What it will do

Render — execute JavaScript with a headless browser for SPA/dynamic pages.
Format — return markdown (LLM-ready), html, raw text, or a screenshot.
Extract — pull structured JSON from a page given a schema.
Unblock — an auto mode that detects a block (challenge page, 403, CAPTCHA) and automatically escalates the IP class and rendering path, then retries.

Planned request shape

The API is designed to be familiar to anyone who has used a modern scrape API:

POST /v1/scrape
{
  "url": "https://example.com/product/42",
  "formats": ["markdown", "json"],
  "onlyMainContent": true,
  "location": { "country": "de" },     // reuses targeting
  "proxy": "auto",                      // datacenter → residential → mobile
  "waitFor": 2000,
  "schema": { "title": "string", "priceUSD": "number" }
}

location and proxy map directly onto the proxy layer’s targeting and IP-type escalation, so the scrape layer is a thin, well-understood wrapper over egress you control.

Why on bytetourist

Scraping APIs buy their proxy bandwidth at significant cost. Running extraction on your own fleet means:

one bill, not two (no upstream proxy markup);
the same routing intelligence (freshest_ip, geo, sessions) under the scraper;
caching and structured-extraction results you own.

Caching & cost (planned)

A response cache (maxAge) will let repeat fetches of the same URL return a recent result instead of re-egressing — cutting both latency and bandwidth for high-volume crawls.

Want this sooner, or have a specific target in mind? Open an issue on GitHub.