llms.txt Generator
Build an AI-readable index of your site
Scan your website
We read your sitemap first, then fall back to following internal links. Your robots.txt is respected.
Advertisement
Banner adResponsive
llms.txt Generator – Create an AI-Friendly Site Index
Create an llms.txt file for your website. Scan your sitemap or pages, organize important links into sections, check the result, and export a clean Markdown file.
Related tools
What is llms.txt Generator?
Create an llms.txt file that gives AI systems and other automated agents a concise, human-readable overview of the important content on your website. Enter a site URL and the tool reads your sitemap (falling back to following internal links), respects robots.txt, and skips noindex pages. You can then review and organize the results, edit titles and descriptions, add your own links and context, and check the file against the llms.txt format before copying or downloading it, ready to publish at your site's /llms.txt path. Prefer not to scan? Build the file manually instead.
- llms.txt
- ai agents
- llm
- seo
- sitemap
- robots.txt
- ai search
Features
What you get with llms.txt Generator.
Automatic Page Discovery
Reads your sitemap, including nested sitemap indexes, and falls back to following internal links when there isn't one.
Respectful Crawling
Honors robots.txt rules, skips noindex pages, and prefers each page's canonical URL so the file lists clean, indexable links.
Scan Controls
Limit the number of pages and exclude paths with simple patterns, so large sites scan quickly and stay focused.
Organize & Edit
Group links into sections, reorder them, edit titles and descriptions, filter the list, and add links or context of your own.
Built-in Format Checks
Get errors and warnings for missing titles, invalid or duplicate URLs, and oversized files before you publish.
Clean Markdown Output
Copy or download a structured llms.txt with a title, summary, optional context, and Markdown link sections, with an “Optional” section placed last.
Advertisement
In-feed adResponsive
Frequently asked questions
Common questions about llms.txt Generator.
llms.txt is a proposed convention for publishing a concise, human-readable Markdown file at a website's root. It gives AI systems and other automated agents an overview of the important pages and resources on a site.
Place the file at the root of your website so it is available at /llms.txt. For example, a site at https://example.com would publish it at https://example.com/llms.txt. In a typical Next.js application, put the file in the public/ directory.
The generator reads robots.txt for sitemap locations, then checks common sitemap paths and follows sitemap indexes. If it can't find a usable sitemap, it follows same-site links starting from your homepage. Pages blocked by robots.txt or marked noindex are skipped, and you can exclude more paths yourself. Everything is editable before you export.
In the llms.txt format, a section named “Optional” marks links that are secondary and can be skipped when an AI system needs a shorter context. This tool always places a section with that name at the end of the file.
Some sites block automated requests, sit behind a login, or rely on client-side rendering that hides links from a simple crawler. If a scan returns nothing useful, use “Build it manually” to create the file yourself, or lower the page limit and exclude large sections.
No. llms.txt is a proposed community convention rather than an established web standard, and AI systems may or may not use it. It does not replace robots.txt or sitemap.xml; those files serve different purposes.
The generator fetches your public pages to build the list. Scan results may be cached briefly to speed up repeat requests for the same site, but your edits and the final file stay in your browser and aren't saved to an account.
Advertisement
Mobile banner adResponsive
Comments
Share feedback, ask questions, or help others use this tool.
Join the discussion
Sign in to comment, reply, and report inappropriate content.