llms.txt Is the New robots.txt — Why Every Developer Should Care

For 30 years, robots.txt has been the handshake between websites and crawlers. "Here's what you can index. Here's what you can't." Now there's a new file showing up in website roots: llms.txt . And it flips the script entirely. Instead of telling bots what not to read, llms.txt tells AI what it should read — a curated, structured summary of your site designed specifically for language models. This just hit the top of Hacker News with 800+ points, and the implications are bigger than most people realize. What Is llms.txt? It's a markdown file at your site root ( /llms.txt ) that provides: # Your Site Name > A one-line description of what this site/project is. ## Docs - [ Getting Started ]( https://yoursite.com/docs/getting-started.md ) : Quick setup guide - [ API Reference ]( https://yoursite.com/docs/api.md ) : Full API documentation - [ Examples ]( https://yoursite.com/docs/examples.md ) : Code examples and tutorials ## Optional - [ Blog ]( https://yoursite.com/blog/index.md ) : Lates