How to Add Web Browsing to Your RAG Pipeline

You're building a RAG application. Your LLM needs fresh information from the web. ChatGPT has web browsing built in, but your custom pipeline doesn't. Options: Google Custom Search API — $5/1,000 queries, returns URLs only (not content) SerpAPI — $50/month, still just URLs Scrape it yourself — build HTTP client, handle JS rendering, parse HTML, extract content... Use a purpose-built tool — this is what I built The Solution: RAG Web Browser I built a web browser specifically for RAG pipelines. It searches Google, scrapes the results, and returns clean Markdown. How It Works Search Query -> Google SERP -> Top N URLs -> Fetch HTML -> Readability -> Markdown You send a search query ("best python frameworks 2026") It queries Google via SERP proxy Fetches the top N result pages Extracts main content using Mozilla Readability (same as Firefox Reader View) Converts to Markdown — compact, LLM-friendly, preserves structure Python Example from apify_client import ApifyClient client = ApifyClient