5 GitHub Actions Workflows I Use to Run Free Web Scrapers, Monitors, and Data Pipelines

GitHub gives you 2,000 free CI/CD minutes per month. Most developers use them only for tests and deploys. I use them to run web scrapers, data pipelines, and monitoring scripts. Here are 5 workflows you can steal. 1. Daily Data Scraper Scrape any public data source and commit results to your repo: name : Daily Scrape on : schedule : - cron : " 0 6 * * *" # 6 AM UTC daily workflow_dispatch : jobs : scrape : runs-on : ubuntu-latest steps : - uses : actions/checkout@v4 - uses : actions/setup-python@v5 with : python-version : " 3.12" - run : pip install httpx - run : python scraper.py - name : Commit data run : | git config user.name "Bot" git config user.email "bot@example.com" git add data/ git diff --cached --quiet || git commit -m "data: $(date -u +%Y-%m-%d)" git push Your scraped data lives in the repo's git history. Free version control for your data. 2. Multi-Source Aggregator Scrape 5 sources in parallel using matrix strategy: name : Aggregate Sources on : schedule : - cron : " 0 *