v0.1.8 · Yozh Crawler is here

Easy web scraping
for AI agents
and humans

One open-source tool for data collection and scraping. Start from a single URL, crawl the site, and get structured data as pages load. Works with MCP and tools like Claude and Cursor, with ready presets for marketplaces like Amazon and eBay.

Install from GitHub Read the docs

84 Stars on GitHub

MIT License · Free forever

v0.1.8 Latest release

Open Scraper Tester

Scraper URL http://localhost:8000 Check ok Crawler URL http://localhost:8001 Check ok CyberYozh API Docs ↗ Get API Key ↗

Scrape Page

Batch Scrape

Crawler

Jobs

MCP

Crawl Parameters

Seed URL *

Scope Mode

same-domain

Max Depth

Max Pages

Per-domain RPS

1.0

Per-domain

Include Patterns (regex, one per line)

^https://amazon\.co\.uk/dp/

Exclude Patterns (regex, one per line)

/cart|/account|/checkout

Crawl Proxy⚠ crawl proxy=none — no IP rotation

Proxy Type

none

Enable scraping (keep raw_html / screenshot / extracted data per page)

Result

crawl_________________ · status: idle · visited: 0 / 30 · queued: 0 · failed: 0 · dedup: 0

Crawl progress and pages will appear here...

What's new · v0.1.8

Crawl full websites and get data in one run

Yozh Crawler maps the full site structure and processes pages as they are found. You get both coverage and usable results in a single run, without managing separate crawl and scrape steps. It works directly inside your existing data collection and scraping workflows.

Yozh Crawler, integrated NEW

Start from one URL and map the whole site. Pages are discovered and processed in one run, so you get full coverage without extra steps.

Discovery mode NEW

Only need links? Switch to discovery mode and map the site without parsing. Faster, lighter, and useful for audits and structure checks.

Stealth browser

Matches headers, location, and browser signals to your proxy. Helps keep sessions consistent across different regions.

Batch scraping

Run hundreds of URLs in one request. Process everything in parallel with one clean output.

Two-stage cancel NEW

Stop jobs without breaking results. Let active pages finish or stop everything instantly when needed.

MCP integration

Connect directly with tools like Claude and Cursor. Use scraping and crawling as part of your existing workflows.

10 built-in presets

Amazon, eBay, Walmart, Google, Google Shopping, YouTube, LinkedIn, Bing, Yandex — pass the source name and parameters, get a normalized response. No selectors to write.

LLM self-heal NEW

A site ships a layout change and your CSS selectors return empty? The parser asks an LLM (OpenAI / Anthropic / Gemini / OpenRouter) to regenerate the selector on the fly. Pipeline keeps running.

AI-generated presets NEW

Need a site we don't ship a preset for? Describe what to extract and paste a sample URL — POST /api/v1/presets/generate returns a ready preset JSON.

Authenticated sessions NEW

Log in once via a declarative DSL (goto / fill / click / wait) or paste exported cookies. Reuse the session in every scrape. Required for walled targets like the LinkedIn preset.

Interface

Simple scraping infrastructure under your control

Set up your workflow in one place and run it the way you need. Everything is clear, organized, and easy to manage without extra tools.

Built-in proxies, ready to use

Access residential, mobile, and datacenter proxies in 120+ countries. Over 50 million IPs with 99.9% uptime. Includes fingerprinting support to align device, browser, and network behavior for stable sessions and better control.

Flexible data extraction

Use CSS selectors, XPath, or auto-detect for popular sites. Get clean, structured JSON without complex setup.

All data in one place

Get raw HTML, screenshots, and parsed data together. Everything you need, without switching between tools.

Simple control and MCP access

Stop jobs anytime with one click and switch between scraper and crawler. Your workflow stays simple and in your control.

MCP · Model Context Protocol

AI agents built for web scraping and crawling

Connect Yozh Crawler and Yozh Scraper to Claude, Cursor, or any MCP client. Switch tools with one toggle, no extra code required.

Web scraping features

All Yozh Scraper features are available as functions for your agent, including scraping, batch runs, and screenshots.

One minute to connect

Copy the config, restart your client, and you're ready.

Works with any MCP client

Claude Desktop, Cursor, Cline, or custom agents.

Full control of the toolkit

Choose how your agent runs tasks, from single pages to full batches.

Run it your way

Use a local MCP server or deploy it on your own infrastructure.

// ~/Library/Application Support/Claude/claude_desktop_config.json
{
  "mcpServers": {
    "yozh-scraper": {
      "type": "http",
      "url":  "http://localhost:8000/mcp"
    },
    "yozh-crawler": {
      "type": "http",
      "url":  "http://localhost:8001/mcp"
    }
  }
}

// Cursor → Settings → MCP → Add new MCP Server
{
  "mcpServers": {
    "yozh-scraper": {
      "type": "http",
      "url":  "http://localhost:8000/mcp"
    },
    "yozh-crawler": {
      "type": "http",
      "url":  "http://localhost:8001/mcp"
    }
  }
}

// Cline → MCP Servers → Edit cline_mcp_settings.json
{
  "mcpServers": {
    "yozh-scraper": {
      "type": "http",
      "url":  "http://localhost:8000/mcp",
      "disabled": false
    },
    "yozh-crawler": {
      "type": "http",
      "url":  "http://localhost:8001/mcp",
      "disabled": false
    }
  }
}

// Anthropic Messages API · MCP via Streamable HTTP
// Note: scraper must be reachable from Anthropic's servers (deploy publicly)
import Anthropic from "@anthropic-ai/sdk";

const response = await new Anthropic().messages.create({
  model: "claude-opus-4-8",
  mcp_servers: [
    { type: "url", url: "https://your-host/mcp", name: "yozh-scraper" },
    { type: "url", url: "https://your-host:8001/mcp", name: "yozh-crawler" }
  ],
  messages: [...]
});

Full MCP integration guide

Yozh Crawler · v0.1.8

Crawl, stream, and parse pages in real time

Yozh Crawler starts from one URL, discovers pages across the site, and processes them as they load, giving you full structure and usable data in one run.

Live streaming

Pages appear as they are discovered and processed. Connect directly from your client, agent, or pipeline without waiting for full runs.

Discovery mode

Only need links? Map the site without parsing. Faster and more cost-efficient for structure checks and audits.

Two-stage cancel

Stop jobs safely. Let active pages finish or stop everything instantly when needed.

MCP target switch

Switch between scraper and crawler with one toggle. Your agent uses the right tool for each task.

Live site tree in the UI

See the site structure build in real time. Follow progress page by page without leaving the dashboard.

Result

crawl_a8f3c124b6e0d7 · status: idle · visited: 0 / 30 · queued: 0 · failed: 0 · dedup: 0 · out-of-scope: 0 · retries: 0

▶ Site Map (0 pages)

URL	Parent	Depth	Status	Took

Try Yozh Crawler See full changelog

Combo · Crawler + Scraper

From one URL to full site coverage

The crawler finds pages, the scraper processes them, and your AI agent uses the results, built for both beginners and experts.

Crawler

Discovers all URLs

Yozh Scraper

Extracts to JSON

AI agent

Analyzes & decides

Competitive analysis

Track competitor catalogs, prices, and product details in one place. Find gaps in your own listings and respond faster to market changes.

Crawler JSON AI

Price monitoring

Track price changes across marketplaces automatically. Set alerts and get notified in Telegram or Slack when prices move.

Batch Schedule Alerts

ML training data

Collect clean, structured datasets from public sources, ready for analysis or model training. Built for reliable data collection at scale.

Stealth Proxy JSON

Marketplace & site presets

Start scraping right away with ready-to-use presets

Locales, currencies, selectors and anti-bot handling — already configured. Pick a preset to see what gets extracted.

10 built-in presets · LLM self-heal when a site layout changes · generate your own from a sample URL Request a preset →

Every Amazon listing, normalized

All fields below are extracted automatically — no selectors to write.

Need a site we don't cover yet?

Open an issue on GitHub or contribute your own preset — we review PRs within a week.

Open an issue

Proxies

Proxies built for scraping

Three types, six configurations. Pick by job, speed, and budget.

Mobile 4G/5G

Real mobile devices on carrier networks. The highest trust tier with strict anti-bot systems.

Private dedicated from $1.7/day

Personal device, manual IP rotation via API

Shared from $0.9/day

One device, multiple users

Learn more about Mobile proxies

How it works

Each IP comes from a real smartphone or LTE/5G modem connected to a major mobile carrier. The IP is shared by thousands of regular subscribers, so blocking it would block real users — which is why anti-bot systems trust mobile IPs the most.

Best for

Account management on social platforms
Aggressive anti-bot targets (Cloudflare, PerimeterX)
Affiliate work and ad verification
Tasks requiring sustained "human-like" trust

Specs

4G LTE and 5G networks worldwide
UDP support, VPN VLS connectivity
Manual IP rotation via API or dashboard link
OS fingerprint adjustment
Unlimited bandwidth on most plans

Recommended for scraping

Residential

IPs from real home internet users across 120+ countries. Access to 50M+ IPs with a strong balance of trust and cost.

Static dedicated from $5.29/mo

Real routers, fixed IP for the whole rental period

Rotating from $0.9/1Gb

100M+ IP pool, auto-rotate by timer or per request

Learn more about Residential proxies

How it works

Residential IPs are sourced from real home internet subscribers who consent to share bandwidth in exchange for compensation, through a transparent opt-in network. Their connection looks like any other home user — because it is one.

Best for

Web scraping at scale (the sweet spot)
Marketplace data collection
SEO research, SERP scraping
Brand and price monitoring
Ad verification campaigns

Specs

120+ countries, 100M+ unique IPs in rotating pool
Choose static or rotating based on the task
City and ASN targeting available
HTTPS, SOCKS5 protocols supported
Sticky sessions up to 30 minutes

Datacenter

Maximum speed, minimum cost. 2M+ IPs across data centers worldwide.

Dedicated static from $1.9/mo

One IP assigned to you, unlimited traffic

Shared static from $0.5/mo

Shared pool, the budget option for simple jobs

Learn more about Datacenter proxies

How it works

IPs are hosted in data centers, not connected to home or mobile users. They're cheaper and faster than residential, but anti-bot systems can identify them as non-human, so use them where trust isn't critical.

Best for

High-volume scraping of friendly sites
Internal tooling and load testing
API endpoints with no IP-trust filters
Tasks where speed matters more than disguise

Specs

2M+ IPv4 and IPv6 addresses
Dedicated or shared configurations
Unlimited bandwidth on dedicated plans
HTTPS, SOCKS5 protocols supported
Lowest latency of all three types

Choose proxies for your use case → Sign in required

Ethics · Non-negotiable

Ethics is built into how we run our platform

Open-source code is just the starting point. How data is collected and how proxies are sourced matters just as much. These are the standards we follow in practice, not statements on a page.

Ethical data collection

We build tools, not loopholes. Yozh Scraper is engineered to make legitimate, public-data scraping reliable — not to break things that should stay closed.

Publicly accessible data only — no logged-in scraping by default
Built-in rate limits prevent server overload
robots.txt and sitemap respect by default (overridable)
No PII collection or storage — neither by us, nor by default for users
Tooling that helps you stay GDPR and CCPA compliant

Transparent proxy network

Where residential IPs come from is the honesty test for every proxy provider. Here's where ours come from — explicitly:

Opt-in network. Real users consent and earn for sharing bandwidth
One-click opt-out at any time, no questions asked
Audited supply chain with documented sourcing
Zero malware, zero hidden SDK injection — ever
Mobile IPs from devices we own and operate, not from compromised handsets

Fast abuse response

If someone misuses our infrastructure to harm others, we want to know — and act on it before it gets worse.

Direct line to a real human, not a ticket queue
Transparent investigation process for every legitimate report
Active cooperation with law enforcement on confirmed cases
Public abuse policy, not buried in fine print

abuse-reports@cyberyozh.com

Average response time: 45 minutes

If a use case involves misuse or harm, our tools are not built for it. We prioritize responsible use over customer growth.

If a use case requires hiding identity to harm others, our tools aren't for it. Period. We'd rather lose the customer than the principle.

Our policies and standards

Clear rules that guide how we operate, from data collection to proxy sourcing and platform use. These are the actual policies our team follows, not summaries.

Use policy

Defines what is allowed on our infrastructure and what is restricted, with clear reasoning.

Abuse response policy

Explains how reports are reviewed, handled, and resolved, including response times.

Network sourcing standards

Details how residential and mobile IPs are sourced, with an opt-in model and verified supply.

Customer verification policy

Covers onboarding, when verification is required, and how user data is handled.

Restricted targets

Lists blocked domains and use cases, including sensitive sectors like government and finance.

Data handling policy

Explains what data is collected, what is not stored, retention limits, and third-party sharing.

In production

How teams use
CyberYozh Data

Real-world deployments

Web scraping & analytics

Automate large-scale collection of public data: product catalogs, ratings, reviews, geo-data. Yozh Scraper plus residential proxies — legal, stable, and clean.

AI agents & automation

Build autonomous agents that scrape, analyze, and act. MCP integration turns Yozh Scraper into a native tool for Claude and Cursor — no custom integration needed.

E-commerce & marketplaces

Track prices, inventory, and product availability across Amazon, eBay, and the rest of the world's marketplaces. Presets save weeks of engineering.

Honest takes from independent sources

All articles

AFFMaven review of CyberYozh's proxy infrastructure and multi-accounting platform

AFFMaven Reviews CyberYozh's Proxy & Multi-Accounting Platform

"CyberYozh stands out as a powerful platform for affiliates and arbitrage teams, combining reliable proxy infrastructure, identity tools, and strong support in one ecosystem."

CyberYozh App B2B platform for global digital operations and network infrastructure

TechBullion Covers CyberYozh's B2B Platform Expansion

"CyberYozh App enables businesses to launch, test, and operate digital products across more than 30 countries through a scalable infrastructure built for global connectivity, verification, and digital operations."

CyberYozh App infrastructure for enterprise proxy workloads and global connectivity

Grit Daily Highlights CyberYozh's Infrastructure for Enterprise Proxy Workloads

"Built for organizations that rely on proxies every day, CyberYozh App delivers infrastructure designed for stability, scalability, and high-volume workloads rather than one-off sessions."

Loved by data teams and AI builders

What people building with Yozh say

5.0 / 5 · 5 reviews

GitHub

Swapped our in-house Playwright cluster for Yozh in an afternoon. The MCP endpoint dropped straight into our Claude agent — zero glue code, crawler and scraper just worked.

Marcus Reinhardt Lead Data Engineer Northwind Analytics May 18, 2026

The preset system is the killer feature. We pass a source name and get clean JSON back; the self-heal even caught two Amazon layout changes before we noticed.

Priya Nair Founder ScrapeStack April 29, 2026

Finally an open-source scraper that treats proxies and sessions as first-class. We run it behind login walls for partner portals — sessions persist and results stay consistent across regions.

Daniel Osei Backend Engineer Loopfeed April 3, 2026

Connected it to Cursor over MCP and my agent pulls live web data mid-task now. The streaming crawl over SSE is exactly what agent workflows were missing.

Elena Kovac AI Engineer Vektor Labs June 11, 2026

GitHub

We moved off a paid scraping API to cut costs and braced for a downgrade — got the opposite. Self-hosted, no per-request billing, and the output schema is cleaner than what we used to pay for.

Sofia Almeida Engineering Manager Tabbly February 22, 2026

About web scraping, crawling, and AI agents

Blog and articles

All articles

Web scraping

Yozh Scraper

Web scraping 08/14/2026

Web scraping for lead generation: Complete guide for 2026

A practical guide to building a reliable lead generation web scraping workflow, from finding public business data to crawling, extraction, validation, proxy selection, and CRM-ready output.

Web scraping 08/14/2026

Best Amazon product ranking data scraping tools and techniques for 2026

A practical guide to collecting Amazon search rankings, Best Sellers Rank, reviews, prices, and competitor data with CyberYozh Data while preserving marketplace, location, timestamp, and ranking…

Resilient Scraping Pipelines: Introducing LLM Self-Healing Parsing in Yozh Scraper

Data extraction is the foundational layer of modern artificial intelligence development, market intelligence, and competitive analysis. However, building reliable data pipelines remains an engineering bottleneck. Target…

9 min

Roman

Changelog

Web scraping changelog and updates

Yozh Scraper is open-source, with all updates available on GitHub. Here are the latest highlights.

v0.1.8 10 August 2026 Latest

v0.1.8

fingerprint_profile on POST /scrape/page, POST /scrape/pages and the crawler/search scrape_options: pin the Camoufox fingerprint's OS and WebGL vendor instead of Camoufox's per-launch random draw (which claimed an OS the server is not two launches in three). Profiles: auto (default, resolves to CAMOUFOX_FINGERPRINT_PROFILE, ships windows_on_host), windows_on_host, host, random, and the bare names windows / macos / linux. The old spoof_os keeps working and equals the three bare names; a caller stating both must not name conflicting operating systems. meta.applied_fingerprint reports what actually ran, including when a profile degraded. New env vars CAMOUFOX_FINGERPRINT_PROFILE (default windows_on_host) and HOST_GPU_VENDOR.
Extraction warns when a post_process pipeline nulls an entire column (every row) — a fourth class of silent failure the invalid/empty/length-mismatch selector checks did not cover. Reported only for a fully-nulled column with more than one row, or a scalar field; an optional value missing on a single row stays quiet.
GitHub Actions CI: pylint and pytest -m "not e2e" on every push and pull request, the checks the repo already defined but never enforced.
run_scrape now returns a typed envelope (ScrapeOk / ScrapeErr) built as the same pydantic models the API validates on the way out, with mypy over the queue surface. An older worker meeting a metadata value a newer API introduced degrades that field and keeps the fetched page, rather than dropping it. Internal refactor; the public HTTP response is unchanged.
session_id, cookies and render are now rejected with 422 on the Camoufox engine instead of being accepted and silently ignored (which returned a logged-out or non-rendered page that read as the site having changed). /search maps the rejection to a 400 rather than a 500.

v0.1.7 4 August 2026

v0.1.7 — Pin mcp<2.0 so a fresh image build starts

mcp 2.0.0 made Server.__init__ keyword-only, which fastapi-mcp 0.4.x still calls positionally, so a clean docker build shipped services that raised TypeError in create_app() before serving anything. Both requirements.txt and yozh-crawler/requirements.txt now pin mcp>=1.28.1,<2. The floor keeps the fix for GHSA-vj7q-gjh5-988w (HIGH — WebSocket Host/Origin validation). Drop the ceiling once fastapi-mcp ships a release built against mcp 2.x.

v0.1.6 29 July 2026

v0.1.6 — Extraction/preset fixes (eBay, Walmart, Yandex, price) + selector-timeout classification

Preset extraction repaired for current site layouts:
eBay search — new s-card layout; price regex hardened against the was-price / prefix-class / nested-tag traps.
Walmart — recovered price and rating extraction (anchored so a strikethrough "was" price can't win); dropped google_shopping's dead urls field.
Yandex search — one row per organic block, so titles, links and snippets no longer drift out of alignment (a title could previously come back paired with the wrong result's link).
Price parsing no longer lets a label before the price swallow it: a leading digit is now required, so From $19.99 / Now 19.99 parse the number rather than the label. Currency symbols, thousands separators (US and EU), leading decimals and negatives still parse as before.

v0.1.5 24 July 2026

v0.1.5 — Queue goto-timeout retry fix + dependency security updates

The queue no longer retries a slow page as if the proxy were bad. A navigation (goto) timeout — the page was simply slow, the proxy is fine — used to burn the retry budget and rotate the proxy; it now gets one rotation at most and otherwise returns a timeout to the caller instead of looping. Genuine proxy failures (connection reset, tunnel/auth errors, net::ERR_*) still rotate and retry as before.
Dependency bumps clearing the outstanding advisories in the auxiliary tooling: examples/ (langchain-anthropic) and the local scraper-tester dev harness (express, qs, http-proxy-middleware, follow-redirects). No shipped scraper runtime or API change.

v0.1.0 9 July 2026 First release

v0.1.0 - Durable queue, Camoufox engine & CyberYozh proxy v2

New required services. Browsers no longer run inside the API process. You must now run the redis and scraper-worker services (both are already in docker-compose.yml). Scale capacity with docker compose up -d --scale scraper-worker=N.
POST /scrape/page and /scrape/pages can now return 503 queue_full when the Redis stream depth exceeds QUEUE_MAXSIZE (previously the in-memory queue was unbounded).
Removed env vars (ignored, warned at startup if set): JOB_TIMEOUT_MS → use PAGE_TASK_TIMEOUT_S; JOB_RESULT_MAX → eviction is now a native Redis TTL (JOB_RESULT_TTL_S); JOBS_ENABLED → the queue is always on.
New env vars: REDIS_URL, PAGE_TASK_TIMEOUT_S, LOGIN_TASK_TIMEOUT_S, LOGIN_RESULT_GRACE_S, RECLAIM_IDLE_S, JOB_RESULT_TTL_S, BROWSER_MAX_PAGES, BROWSER_IDLE_SHUTDOWN_S, WARMUP_DWELL_MS.
The in-process job queue and browser worker pool are replaced by a taskiq stream on Redis. The web-scraper container only enqueues page tasks and reads results; browsers run in separate, horizontally scalable scraper-worker containers.

Full release notes on GitHub

Quick start

Get started with web scraping in seconds

Three commands and you have a running scraper + crawler with HTTP and MCP endpoints.

1 Clone

git clone https://github.com/\
  CyberYozh-data/yozh-scraper
cd yozh-scraper

2 Run

cp .env.example .env
docker compose up --build

3 Scrape

curl -X POST localhost:8000/api/v1/scrape/page \
  -H 'Content-Type: application/json' \
  -d '{"url":"https://example.com"}'

Full documentation Plug into Claude / Cursor via MCP →

Contact · Support

Get in touch

A real human reads everything that comes in. Pick the channel that fits — Telegram is the fastest.

Office

Jurija Gagarina 231/329

Novi Beograd · Serbia

Support · Telegram

@CyberYozh_support_bot

Support · Email

support@cyberyozh.com

Report abuse · 45 min avg

abuse-reports@cyberyozh.com

Two-tier support, near round-the-clock First-line on duty almost 24/7. Senior engineers handle the hard cases.

Open Source · MIT License · 84 ★

Ready to scrape?

Yozh Crawler + Scraper is free. Forever. Star us if it helps — every star counts.

Install from GitHub Star on GitHub

Yozh Crawler + Scraper is distributed under the MIT license. Use it. Fork it. Build with it.

Easy web scrapingfor AI agentsand humans

Crawl full websites and get data in one run

Yozh Crawler, integrated NEW

Discovery mode NEW

Stealth browser

Batch scraping

Two-stage cancel NEW

MCP integration

10 built-in presets

LLM self-heal NEW

AI-generated presets NEW

Authenticated sessions NEW

Simple scraping infrastructure under your control

Built-in proxies, ready to use

Flexible data extraction

All data in one place

Simple control and MCP access

AI agents built for web scraping and crawling

Web scraping features

One minute to connect

Works with any MCP client

Full control of the toolkit

Run it your way

Crawl, stream, and parse pages in real time

Live streaming

Discovery mode

Two-stage cancel

MCP target switch

Live site tree in the UI

Result

From one URL to full site coverage

Crawler

Yozh Scraper

AI agent

Competitive analysis

Price monitoring

ML training data

Start scraping right away with ready-to-use presets

Every Amazon listing, normalized

Need a site we don't cover yet?

Proxies built for scraping

Mobile 4G/5G

How it works

Best for

Specs

Residential

How it works

Best for

Specs

Datacenter

How it works

Best for

Specs

Ethics is built into how we run our platform

Ethical data collection

Transparent proxy network

Fast abuse response

Our policies and standards

Use policy

Abuse response policy

Network sourcing standards

Customer verification policy

Restricted targets

Data handling policy

How teams useCyberYozh Data

Real-world deployments

Web scraping & analytics

AI agents & automation

E-commerce & marketplaces

What press and businesses say

Honest takes from independent sources

AFFMaven Reviews CyberYozh's Proxy & Multi-Accounting Platform

TechBullion Covers CyberYozh's B2B Platform Expansion

Grit Daily Highlights CyberYozh's Infrastructure for Enterprise Proxy Workloads

Loved by data teams and AI builders

About web scraping, crawling, and AI agents

Blog and articles

Web scraping for lead generation: Complete guide for 2026

Best Amazon product ranking data scraping tools and techniques for 2026

Resilient Scraping Pipelines: Introducing LLM Self-Healing Parsing in Yozh Scraper

Easy web scraping
for AI agents
and humans

How teams use
CyberYozh Data