Scraping Nonprofit Financial Data: Form 990 and IRS Records

Scraping Nonprofit Financial Data: Form 990 and IRS Records Every nonprofit in the United States is required to file Form 990 with the IRS, disclosing executive compensation, revenue, expenses, and program activities. This data is public but notoriously difficult to access in bulk. Let's build a Python scraper to extract and analyze nonprofit financial data systematically. Why Nonprofit Financial Data Matters Donors, journalists, and researchers need to evaluate whether nonprofits use funds effectively. Form 990 reveals executive pay ratios, fundraising efficiency, program spending percentages, and financial health indicators. Data Sources IRS Exempt Organizations BMF — master list of all tax-exempt organizations ProPublica Nonprofit Explorer API — structured 990 data (best free source) IRS 990 XML files — raw filings on AWS (bulk download) State charity registrations — additional compliance data ProPublica Nonprofit Explorer API import requests import time class NonprofitExplorer : BA