Our llms.txt Has 708 Links. Google Ignores All of Them.

Eight AI tool directories, one request each, on the morning of 23 August 2026: curl https://<site>/llms.txt. Five returned 404. Futurepedia, There's An AI For That, Toolify, FutureTools and aitools.fyi were all empty at /.well-known/llms.txt and /llms-full.txt too, under a research user agent and a stock Chrome one alike. Three served a file: G2, Product Hunt and TopAI.tools. Ours served 55,528 bytes of it.

The gap is not a scoreboard. It is a disagreement about what the file is for. G2's llms.txt describes G2 and hands an assistant the URL patterns to build pages from. HokAI's enumerates 607 entity pages by name and hands over the pages. One is a brochure, the other is an index. And the finding sitting underneath both, which we checked before writing a word of this: the strongest public evidence says hardly anyone reads either kind.

Disclosure: HokAI wrote this about HokAI.

What came back from eight fetches

Link-list entries in each directory's /llms.txt, all fetched 23 August 2026.

The unit on that chart is the link-list entry: one - [name](url) line, which is the exact construct the llms.txt specification uses to point a model at a page. Jeremy Howard published the proposal on 3 September 2024 and issued a v2 revision on 10 August 2026. It asks for an H1 with the site name, a blockquote summary, then H2-delimited sections of those link lists.

Two of the three files that exist are missing from the counts on purpose. TopAI.tools and Product Hunt both serve an llms.txt to a browser and refuse it to an automated re-fetch, so we cannot give you a link that proves a number about them. Rather than print figures you would have to take on trust, we left them uncounted and said so on the chart.

A brochure and an index are different files

G2's /llms.txt, fetched 23 August 2026.

G2's file runs 466 words across six sections, with 11 link lines. Almost none of them point at a product. They point at https://www.g2.com/products/{product-slug}/reviews and nine sibling templates, plus a genuinely clever touch: every category exposes its Grid plot as PNG, SVG and JSON at a predictable path, so an assistant can embed the picture or read the scores underneath it.

That is not a failure, it is a different bet. G2 decided the useful thing to give a model is an explanation and a URL grammar, which is a defensible reading of a spec whose only required element is a heading.

The counts have to match, or the file is fiction

We read it the other way. Our file carries ten sections, and four of them are inventories: 451 tools, 102 models, 54 companies, 87 guides.

HokAI's /llms.txt, fetched 23 August 2026, where the Tools section begins its alphabetical run.

Those four numbers are the whole argument, because of what they equal. Fetch /api/directory/overview on the same morning and it answers 451 tools, 102 models, 54 companies. Fetch /api/guides and count the rows: 87. The file and the tools directory agree to the digit on all four, because neither is typed by hand. Both are rendered from the same records.

The blockquote's "678 entries and counting" is those tabs plus agents, skills and services, and it moves the same day the models tab moves.

A hand-written llms.txt cannot hold that property for a week. Someone adds twelve listings, nobody edits the manifest, and the file starts describing a site that no longer exists. A generated one is a consistency check with a URL attached. If the counts ever drift apart, the file is lying, and anyone can catch it with two curls.

What the record says about who reads them

Now the part a marketing page would leave out, and the part that decides what this file is actually worth.

SE Ranking crawled close to 300,000 domains and published the result on 20 November 2025. Just 10.13% carried an llms.txt, spread almost flat across traffic tiers: 9.88% at the low end, 10.54% in the middle, 8.27% among the largest sites. Then they tested whether the file moved AI citations. It did not.

Their XGBoost model got more accurate when the llms.txt variable was removed, which is what a feature looks like when it is noise. Search Engine Journal's write-up states the conclusion plainly: no measurable link to citation frequency.

Google's documentation is blunter still. Google Search does not use llms.txt, including for its generative features, the file neither helps nor hurts rankings, and Google's AI optimisation guidance tells site owners they can skip creating one.

A survey of request logs across roughly 900 domains between 4 September 2025 and 13 April 2026 found 1,227 hits on llms.txt-family files. A commercial data aggregator accounted for 794 of them and human browsers for 392. Frontier lab crawlers, meaning GPTBot, ClaudeBot, PerplexityBot and Google-Extended, accounted for zero.

Those names are worth reading slowly. GPTBot belongs to OpenAI, ClaudeBot to Anthropic, Google-Extended to Google: the labs behind the models most people are actually asking, GPT-5.4 and Claude Sonnet 5 among them. Between them they made none of those 1,227 requests.

Then why ship one

Because the version that costs nothing is the version worth having, and the reason is internal to the build rather than external to the crawl.

The file is a projection of the same records that render every listing: the six page types we split entities across, the sources we open and re-check, the policies printed at the top of it. Generating it costs one template. Maintaining it costs nothing, because there is nothing to maintain. And it yields a plain-text artefact whose numbers have to agree with the API, which is a test we get for free.

If the citation picture changes, if a lab says its crawler ingests these files and a controlled study shows lift, we are already shipping the shape that would benefit, and it will have cost a template either way. That is the whole business case. Anyone selling llms.txt as a ranking tactic in August 2026 is selling something the evidence does not support.

The limits of a file that describes itself

Worth being precise about what this survey did and did not establish. It measured presence and shape: whether a file exists, how long it is, how many link entries it carries. It did not measure whether a single assistant acted on any of them. That is not observable from outside, because it needs server logs, and the logs that have been published say the answer is close to nothing.

There is also a structural ceiling that no amount of good formatting clears. A self-declared manifest cannot rank sites against each other, because every site writes its own and every one of them claims to be the good one. Google's search relations people have made exactly this point, and on it they are right.

An index like ours can at least be checked against an API, which is a weaker guarantee than an audit and a stronger one than a paragraph of self-description. It is still a site talking about itself.

And a brochure llms.txt is genuinely the right call for some publishers. If what a model most needs from you is how to interpret your scoring, or the URL grammar for a catalogue far too large to list, then describing beats enumerating. G2 has millions of reviews and a Grid methodology to explain, and a 700-line list of product slugs would serve nobody. Enumeration only pays when the list is small enough to serve whole and stable enough to be worth pointing at.

Four commands, if you want to check this page

The head of HokAI's /llms.txt, fetched 23 August 2026.

Count our link lines, read the directory API, count the guides, then try Futurepedia's file. The first should print 708. The second should show 451 tools, 102 models and 54 companies. The third should return 87 guides. The fourth should 404, and if it ever stops, that is Futurepedia shipping one and this survey going out of date the same day.

The claim here is a small one and it is checkable, which is the only kind worth putting on a page about ourselves. If you would rather skip the plumbing and get a shortlist instead, Smart Match reads the same records this file does and will not ask you to run a single curl. And if any of those four commands disagrees with what you just read, tell us, because the correction goes on this page and stays there for 30 days.

Frequently asked questions

Does adding an llms.txt help you rank in AI search?

No, on the evidence available in August 2026. SE Ranking tested close to 300,000 domains and found no correlation between publishing the file and how often a domain is cited by AI systems; removing the variable made their prediction model more accurate, not less. Google's documentation states that Google Search does not use llms.txt, including for its generative features. Treat anyone selling it as a ranking tactic with suspicion.

Do ChatGPT and Claude actually read llms.txt files?

There is no published evidence that they do. A survey of request logs across roughly 900 domains between September 2025 and April 2026 recorded 1,227 requests for llms.txt-family files, and none of them came from GPTBot, ClaudeBot, PerplexityBot or Google-Extended. Most of the traffic was a commercial data aggregator and ordinary human browsers. No frontier lab has said its crawler ingests the format.

What is inside HokAI's llms.txt?

Ten sections and 708 link-list entries, fetched 23 August 2026. Four of those sections are inventories: 451 tools, 102 models, 54 companies and 87 guides, each listed individually by name and URL. The rest cover editorial policy, security contacts, Smart Match, the core indexes and documentation. The file is generated from the same database rows that render the site.

How is llms.txt different from a sitemap or robots.txt?

Robots.txt tells crawlers what they may fetch, and an XML sitemap lists every indexable URL for search engines. An llms.txt is markdown meant to be read by a language model, offering a curated subset with human-readable notes rather than an exhaustive machine list. HokAI's sitemap carried 1,131 URLs on 23 August 2026 against the llms.txt file's 708 curated links.

Should I add an llms.txt to my own website?

Only if you can generate it automatically from whatever database already renders your pages. Generated, it costs one template and gives you a consistency check, since its counts must agree with your API. Hand-written, it goes stale the first week someone adds pages without editing it, and you will have built a file that describes a site you no longer run for an audience that has not shown up.

Covered in this guide

• HokAI: HokAI is an editor-curated, AI-only directory at hokai.io. Listing requires a flat submission fee; rankings cannot be bought. Every Pulse update is verified against a primary source.
• Anthropic: Anthropic, founded 2021 by 7 ex-OpenAI researchers, builds Claude and was valued near $965B after its May 2026 Series H round.
• Claude Sonnet 5: Claude Sonnet 5, released by Anthropic on June 30, 2026, hits 82.1% SWE-bench Verified with a 1M-token context window at $3/$15 per 1M tokens.
• Google: Google (Alphabet, NASDAQ: GOOGL), founded 1998, serves 8B+ monthly Search users with Gemini 3.5, 190,820 employees, and $402.84B FY2025 revenue.
• GPT-5.4: GPT-5.4 (OpenAI, March 2026) scores 80% on SWE-bench Verified with a 1M-token context window, priced at $2.50/$15 per 1M tokens.
• OpenAI: OpenAI builds the GPT-5.6 model family (Sol, Terra, Luna), o3, ChatGPT (900M+ weekly users), and the OpenAI API. Closed a $122B round at an $852B valuation in March 2026, the largest private funding round in history.

Sources