Scraper worked on my laptop. Deployed to server and got instant 403s.

Scraper worked on my laptop. Deployed to server and got instant 403s. Wrote a scraper last week for product data. Tested it locally, worked fine. Collected 200 products, zero issues. Deployed to my VPS Friday night thinking I could run it on a cron and forget about it. Saturday morning I check the logs. Every single request: 403 Forbidden. Zero data collected. Fun times. What broke Turns out the target site was checking User-Agent. My laptop had requests with a normal browser user agent because I was using Playwright for something else and had set it globally in my profile. The server? Fresh Ubuntu install. Default Python requests User-Agent looks like this: python-requests/2.31.0 Site took one look at that and said no thanks. Fixed it Added a custom User-Agent to the requests header: import requests headers = { ' User-Agent ' : ' Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36 ' } response = requests . get ( ' https://exa