How to build a free HN data pipeline in 30 minutes

Hacker News is one of the richest sources of signal in tech. New frameworks, hiring waves, shifting sentiment — it all shows up on HN before it hits mainstream. But scraping HN yourself is tedious and fragile. In this tutorial, I'll walk you through building a lightweight data pipeline that pulls structured HN data on a schedule, stores it locally, and runs basic trend detection — all for free. The Data Source We'll use the HN Top Stories actor on Apify , which returns clean JSON for top, new, best, ask, show, and job stories. It handles pagination, rate limits, and retries so you don't have to. Apify's free tier gives you enough compute to run this daily without paying a cent. Step 1: Fetch HN Data Install the Apify client: pip install apify-client Then pull the latest top stories: from apify_client import ApifyClient from datetime import datetime client = ApifyClient ( " YOUR_APIFY_TOKEN " ) run = client . actor ( " cryptosignals/hn-top-stories " ). call ( run_input = { " category "