
How to Scrape News Websites and Build a Custom RSS Feed
Most news sites dont offer RSS anymore. Build your own. Strategy 1: Find Hidden RSS Many sites still have RSS feeds: https://site.com/feed https://site.com/rss https://site.com/feed.xml https://site.com/atom.xml Strategy 2: Google News RSS (Any Topic) https://news.google.com/rss/search?q = YOUR+TOPIC&hl = en Free, no key, 100 articles per query. Strategy 3: Build Custom RSS from HTML const cheerio = require ( " cheerio " ); async function scrapeToRSS ( url , selectors ) { const res = await fetch ( url ); const $ = cheerio . load ( await res . text ()); const items = $ ( selectors . item ). map (( i , el ) => ({ title : $ ( el ). find ( selectors . title ). text (). trim (), link : $ ( el ). find ( " a " ). attr ( " href " ), date : new Date (). toISOString () })). get (); // Generate RSS XML const rss = `<?xml version="1.0"?>\n<rss version="2.0">\n<channel>\n<title>Custom Feed</title>\n ${ items . map ( i => `<item><title> ${ i . title } </title><link> ${ i . link } </link></item>` ).
Continue reading on Dev.to Tutorial
Opens in a new tab



