Markdown pages, are they a good solution?

I already written a post about a HTML to markdown converter solution . In that post I suggested that the functionality on the application level would be a better solution. And in the last week Laravel markdown response and Symfony markdown response bundle popped up. I guess other languages with web frameworks will get similar solutions. I consider those solutions to be partial fixes, because they lack the tools to trim or augment the page content for an LLM to get the data it can act on. If you want to provide LLM content I think the best solution is a backend one instead of a frontend one. I consider fully created HTML as a part of the frontend. The elephant in the room is that webpages are also a human construct. An AI scraper doesn't need to follow human navigation. Page links are useless if it can grep the content. How did we get here? With search engine bots their goal was to find all the pages of a website to put them in a search index and rank them. The purpose of AI bots is to