Crawling GCC Government Documents: What Blocked Me

Building GCC LexAI meant ingesting AI regulation documents from UAE and Saudi Arabia government websites. The tech stack worked fine. The websites did not always cooperate. Saudi .gov.sa blocks non-Saudi traffic entirely This took me a while to accept as the actual explanation. cst.gov.sa , cma.gov.sa , sdaia.gov.sa , and their subdomains return connection timeouts. Not 403s, not redirects — timeouts. I tried from Japan, Malaysia, and the US. Same result every time. The problem isn't the origin country; it's that Saudi government sites appear to block all non-Saudi IP ranges at the network level. Changing your crawler's location doesn't help. Proxies in GCC countries are the theoretical fix, but the practical one is to not depend on primary government URLs at all. What worked: Some agencies publish via CDN subdomains ( cdn.nca.gov.sa ), which resolve from outside Saudi Arabia. For agencies without CDN mirrors, I used documents hosted by OECD, law firms, and academic institutions — the