# Web Scraping API for AI Workflows — MCP Scraper

Source: https://mcpscraper.dev/scraper

348 scraper, connection, and automation tools

# Web scraping API for AI workflows

Send MCP Scraper a public URL and inspect the extracted page content in your AI workflow or application. For multiple pages, set a crawl limit and check which pages succeeded before you use the results.

Start for $1 Read the API docs

Illustrative query result Sample data

Query best hvac company Denver CO

34 PAA questions

10 Organic results

6 AI citations

4 Forum threads

Question How much does it cost to replace an HVAC system in Denver?

PAA

Question Is it cheaper to repair or replace an HVAC system?

PAA

Source thisoldhouse.com › hvac-replacement-cost

PAA source URL

348 scraper and automation tools

10,000 maximum requested pages per crawl

100 maximum requested People Also Ask questions

Live browser watch and takeover

Videos and ads

## Collect structured video and ad-library data.

Explore YouTube channels and search, Facebook and Google Ad Libraries, and Facebook and TikTok video transcription. Pick a surface and inspect the structured result.

Have a public video ID? [Inspect a recorded YouTube Transcript API response](https://mcpscraper.dev/use-cases/youtube-transcript-api#recorded-result) and check the timestamped chunks against the source.

mcpscraper.dev/youtube/harvest — mode: channel, @hvacrschool

H

HVAC School

@hvacrschool · 1.28M subscribers · 386 videos

▶

14:32

How to Choose the Right HVAC System for Your Home

284K views · 3 months ago

▶

9:18

HVAC Maintenance Checklist — Spring Tune-Up Guide

142K views · 6 months ago

▶

21:07

Heat Pump vs Gas Furnace: Which Saves More in 2025?

521K views · 1 year ago

▶

11:44

R-410A Phase-Out: What Technicians Need to Know

98K views · 2 months ago

▶

18:56

Refrigerant Recovery: Step-by-Step for Beginners

67K views · 4 months ago

▶

16:22

Why HVAC Systems Fail: Top 10 Root Causes Explained

310K views · 8 months ago

Google Maps

## Find businesses and inspect available profile details.

Retrieve available services, areas served, hours, reviews, and entity IDs for a known business. For a category or local market search, request up to 50 candidates.

For one verified listing, [inspect a recorded Google Maps review sample](https://mcpscraper.dev/use-cases/google-maps-scraper#reviews). If you are evaluating providers, [compare Outscraper alternatives for collecting reviews](https://mcpscraper.dev/alternatives/outscraper).

Illustrative profile demo

mcpscraper.dev/maps/place

Elite Roofing

Roofing contractor

4.8 out of 5 (312 reviews)

4210 Federal Blvd, Denver, CO 80211

(303) 555-0142

Open · closes 6pm

Services

Roof Replacement Roof Repair Storm Damage Inspection Gutter Installation

Areas served

Denver Aurora Lakewood Westminster

cid 8823019471… · ready for AI Overview / Knowledge Graph matching

A profile lookup can return services, areas served, hours, and entity IDs when those fields are available.

Websites

## Choose a single URL, a crawl, or a hosted browser

Start with one representative URL. Check that the returned content and format fit your workflow before you expand to a crawl.

In this [recorded single-page extraction](https://mcpscraper.dev/blog/competitor-website-analysis/#collect-page-evidence), MCP Scraper captured Firecrawl’s homepage heading and signup action.

If the page needs a click or another interaction to show the content, open a hosted browser and inspect what it captured. In a separate [browser field check](https://mcpscraper.dev/blog/debug-missing-web-scrape-records/#compare-browser), Chrome showed 20 distinct quote records across two JavaScript sandbox pages. Two hosted extraction attempts, one on the first static page and one on the first JavaScript page, failed and returned no records.

Still choosing between providers? [Compare AI web scraping tools by job](https://mcpscraper.dev/blog/ai-web-scraping-tools/) and check what a passing result looks like for each.

When you run a crawl, compare the discovered, attempted, successful, and failed page counts before using the export. Use search or site mapping when you still need to find the URLs.

If your workflow starts with a search query, [inspect a recorded Google SERP API response](https://mcpscraper.dev/use-cases/google-serp-api#sample-result) before choosing which result URLs to open.

Evaluating a page extraction API or a source-specific workflow? See the [Firecrawl vs Apify comparison](https://mcpscraper.dev/compare/firecrawl-vs-apify) to choose what to test first. If you are replacing Firecrawl, [compare alternatives by the workflow you need](https://mcpscraper.dev/alternatives/firecrawl).

The [search results versus page extraction](https://mcpscraper.dev/blog/search-results-vs-page-extraction/) guide shows when each step is useful. To collect questions and check which answers and source links were returned, follow the [People Also Ask scraper guide](https://mcpscraper.dev/blog/people-also-ask-scraper/). For question selection, see [how to use People Also Ask for SEO](https://mcpscraper.dev/blog/people-also-ask-seo/).

Illustrative full-site audit

audit_site example.com · 1,284 pages

1,284 Pages crawled

37 types Issues found

18,402 Internal links

512 Images

|  | Issue | Pages |  |
|---|---|---|---|
| 🔴 | Broken internal links (404) links.brokenInternal | 23 |  |
| 🔴 | Pages returning 5xx errors response.error5xx | 4 |  |
| 🟠 | Missing page title title.missing | 11 |  |
| 🟠 | Missing H1 h1.missing | 19 |  |
| 🟠 | Set to noindex indexability.noindex | 8 |  |
| 🟠 | Missing canonical tag canonical.missing | 27 |  |
| 🟡 | Duplicate title title.duplicate | 34 |  |
| 🟡 | Duplicate meta description meta.duplicate | 41 |  |
| 🟡 | Thin content (under 200 words) content.thin | 52 |  |
| 🟡 | Images missing alt text images.missingAlt | 88 |  |
| 🟡 | Missing schema markup schema.missing | 63 |  |
| 🟡 | Title longer than 60 chars title.tooLong | 29 |  |

Illustrative per-page extraction

Select Formats to compare the same sample heading in HTML, JSON, and Markdown. [Inspect a recorded extraction](https://mcpscraper.dev/blog/data/what-is-an-mcp/first-use-summary.json) or [set up your first test](https://mcpscraper.dev/use-cases/web-scraping-mcp-server#connection).

mcpscraper.dev/extract-url

example.com — full page, 1440×900

Mobile viewport also available.

Full-page screenshots, desktop or mobile viewport.

Video analysis

## Understand a video frame by frame.

Sample frames, analyze them with vision, and transcribe the audio to understand pacing, hooks, visual style, and the production choices behind the result.

0:00

0:10

0:20

0:30

0:40

0:50

1:00

1:10

1:20

1:30

480

max frames

6,667 Credits

per requested 120-frame tier, plus model usage

60 min

longest video supported

Hook

What grabs attention in the first 3 seconds, and why it works.

Pacing & energy

Cut frequency, scene length, and energy curve across the video.

Visual style

Color grading, framing, on-screen text patterns — the recognizable look.

Replicate it

A step-by-step recipe for shooting something in the same style.

Accepts YouTube, Facebook, Instagram, TikTok, or Vimeo links, or a direct video file URL (.mp4/.webm/.mov). Analysis-model usage is billed separately at 1.5× reported cost.

What people build

## A live data backbone for whatever you are building.

Inspect the returned records before handing them to an agent or app. The interactive demos show example workflows.

One MCP Scraper audit of mcpscraper.dev on November 6, 2026 requested up to 200 pages. It discovered 28 URLs and attempted 27, with 26 successful captures and one failure. The job counted one remaining URL without naming it. The failed record, /llms, returned HTTP 200 and a text/plain response the HTML extractor could not use. The [crawl completeness guide](/blog/website-crawl-completeness/) explains how to reconcile those counts.

reddit.com/r/SEO

r/ r/SEO Agent reading

##### How do I check which pages a scraper returned?

Illustrative example

Illustrative workflow. Give agents a hosted browser for permitted sign-ins, clicks, and page reading. Watch the session or take over for a manual step.

Escalation

## Use a hosted browser when the page needs interaction.

Try a direct request first. Use a hosted browser when the page needs rendering, a click, or a sign-in the source allows. Watch the session or take over for a manual step. Check which requested fields or artifacts came back before treating the run as complete.

Example recovery path. Inspect the direct response first. Open a hosted browser if the source permits access and needs interaction. If the result is partial or failed, record what is missing before using it downstream.

Live browser

## Automated when it can be. Drivable when it needs you.

Watch a session live, take over the mouse for a human step, record a replay, reuse saved profiles, and load the Chrome extensions the job requires.

mcpscraper.dev/agent/sessions

checkout-flow · watching preview

Takeover is available inside a hosted session.

Watch a hosted session and take control when the workflow needs a human step.

Use cases

## Choose the web workflow you are building.

Connect a compatible MCP client and confirm it exposes the tool you need. For a direct integration, follow the API docs and verify account access and the returned fields with one small request.

For agent buildersWeb scraping for AI agentsSearch, scrape, map, and recover in a hosted browser with inspectable results.**See the AI-agent workflow →** For MCP clientsWeb scraping MCP serverConnect one remote server and expose typed web-data tools to a compatible client.**See the MCP-server workflow →**

## Choose a scraper test for your workflow

Pick a source, see what to collect, and get a check for the result. The workflow finder helps you choose a first test for Google Search, local businesses and reviews, public webpages, or evidence you want to reuse in another AI session. Your answers stay on the page. The finder does not inspect a website or run a scrape.

[Open the workflow finder](https://mcpscraper.dev/scraper#workflow-finder)

Pricing

## Simple subscriptions with credits included.

Every plan includes credits, parallel browser capacity, and all 348 scraper, connection, and automation tools. Credits last three months.

Start with a representative request and compare the scope you asked for, the results you received, and the settled credit usage. Hosted browser usage is metered for the time the session stays open, including idle time. Close the session when you finish.

Starter

$1first month

Then $12 per month. A practical starting point for individual agents and prototypes.

- 80,000 credits per month
- 3 parallel browsers
- All 345 tools and live browser

Start for $1

Growth

$40/ month

More throughput for products and workflows that rely on live web data every day.

- 266,667 credits per month
- 10 parallel browsers
- All 345 tools and live browser

Get Growth

Scale

$100/ month

More throughput for products and workflows that rely on live web data every day.

- 800,000 credits per month
- 20 parallel browsers
- All 345 tools and live browser

Get Scale

Active integrations are 15,000 Credits/month each, drawn from your balance. Each $5/month pack adds 2 parallel browsers. Contact us for volume, SLA, or invoicing needs. For your first test, compare the Credits used with the pages or records that passed your check. See the [current per-operation rates](https://mcpscraper.dev/pricing#rate-card).

Works wherever you do

## Choose how to run your first scraper test.

Use a compatible AI client, the dashboard, or the API. Confirm access and inspect the result of one small request.

Claude Desktop and Code

Connect MCP Scraper in Claude, then check that the tool you need appears in the client.

Compatible MCP clients

Follow the setup for your client and test a small request before running a larger job.

Dashboard

Run tools by hand and inspect the result without writing code.

API and exports

Call the API directly and check which output formats your selected operation supports.

Install the MCP package

Install the package, then follow the [MCP connection steps](https://mcpscraper.dev/use-cases/web-scraping-mcp-server). For a direct integration, see the [SDK and API documentation](https://mcpscraper.dev/docs#sdk-surfaces).

install.sh

```
npx -y -p mcp-scraper@latest mcp-scraper
```

Start with the whole toolkit

## Give your agent the live web.

Every scraper, connection, automation, and browser surface in one account.

Start for $1
