
Web2API — Turning Websites into REST APIs (and MCP Tools)
I needed data from websites that don't have APIs. Not once, not as a quick scrape, but as persistent, queryable endpoints I could hit programmatically. So I built Web2API . The Problem Most useful data on the internet lives behind HTML. Some sites offer APIs, many don't. The typical approach is writing one-off scrapers — fragile scripts that break whenever the site changes a CSS class. I wanted something different: Declarative — define what to extract, not how to click through pages Persistent — a running service with stable endpoints, not a script I run manually Modular — add new sites without touching the core codebase AI-ready — expose scraped data as tools that language models can call The Solution Web2API is a FastAPI service backed by Playwright (headless Chromium). You define recipes in YAML — each recipe describes a website, its endpoints, and what data to extract. The service runs continuously and serves the scraped data as clean JSON REST endpoints. A Recipe Looks Like This n
Continue reading on Dev.to Python
Opens in a new tab




