Web Scraping in 2026: What Works, What Doesn't, and What's Legal

Web scraping is the automated extraction of data from websites. It sounds simple, and for basic cases it is. Fetch a page, parse the HTML, extract the data you need. But modern websites are increasingly hostile to scrapers, and the legal landscape has become significantly more complex. The technical landscape Static HTML sites. These are the easy case. Fetch the page, parse the DOM, extract data using CSS selectors or XPath. Tools: fetch + cheerio (Node.js), requests + BeautifulSoup (Python). const response = await fetch ( ' https://example.com/products ' ); const html = await response . text (); const dom = new JSDOM ( html ); const prices = dom . querySelectorAll ( ' .product-price ' ); JavaScript-rendered sites (SPAs). The HTML source contains a root <div> and a JavaScript bundle. The actual content is rendered client-side. Traditional fetch-and-parse does not work because the content does not exist in the initial HTML. Solution: headless browsers. Puppeteer (Chrome) or Playwright (