DeepThinking AI

Tag

infrastructure

Covered in AI Engineering, where the background and the sources for this subject live.

AI Engineering

Do robots.txt and llms.txt stop AI agents from writing?

robots.txt, llms.txt, Content-Signal and AIPREF all describe what an agent may fetch. The 2,000 packages uploaded to RubyGems in May went through a signup form and a publish endpoint, which none of those files govern. Read controls are advisory. Write paths need identity and rate limits.

3 min read