AI search · 10 min read

llms.txt and structured data, explained in plain English

What llms.txt is and is not, which JSON-LD schema types a local business needs, and how breadcrumbs, sitemaps and robots.txt help search and AI tools.

llms.txt is a proposed plain-text file that gives AI tools a short, curated map of a website. It is not a standard, and adding one will not by itself make AI assistants recommend you. Structured data in JSON-LD, clean breadcrumbs, an accurate XML sitemap and a sensible robots.txt file do the heavier lifting, because search engines and AI systems already read them.

What llms.txt actually is

llms.txt is a proposal published at llmstxt.org in 2024. The idea is simple: put a Markdown file at the root of your website, at yoursite.com/llms.txt, that tells large language models what the site is and where the most useful pages are. Instead of making an AI tool crawl dozens of pages full of menus, scripts and footers, you hand it a short, clean summary with links.

The proposed format has a few parts. A heading with the name of the site or project. A short summary in a blockquote. Optional paragraphs with more detail. Then sections of links, each with a short note about what the linked page covers. The proposal also suggests offering clean Markdown versions of important pages, and an optional section for links that can be skipped when a tool is short on space.

The original audience was software documentation. A developer asking an AI coding assistant about a library benefits when that library’s docs are available in a compact, readable form. That use case is real, and a number of developer tool companies publish llms.txt files for exactly that reason.

What llms.txt is not

llms.txt is not an official web standard, and it is not a ranking factor. Google has said publicly that it does not use llms.txt for Search or AI Overviews. The major AI companies have not committed to reading it as part of how they choose which businesses to recommend. Some crawlers fetch the file when it exists, but fetching a file and relying on it are different things.

It is also not robots.txt. robots.txt tells crawlers what they may and may not access. llms.txt does not grant or deny anything; it is a guide, not a gate. If you want to control access, robots.txt and your server settings are where that happens.

So should a local business have one. It is cheap to add, it does no harm, and it forces a useful exercise: writing down in plain words what your business does and which pages matter most. We include one on the sites we build for that reason. Just do not mistake it for the work that actually moves visibility, which is everything else in this guide.

  • It is a proposal, not a standard
  • It does not block or allow crawlers
  • Google does not use it for Search
  • It is useful as a clean summary, not as a ranking shortcut

Structured data: what JSON-LD does

Structured data is a way of labeling the facts on a page so machines do not have to guess. A human reading your contact page knows which string of digits is your phone number. A crawler can usually figure it out, but structured data removes the guesswork by saying, in a fixed vocabulary from schema.org, this is the business name, this is the telephone, these are the hours.

JSON-LD is the format Google recommends for this. It is a small block of code in the page that visitors never see. It does not change how the page looks, and it should only describe things that are actually on the page or true of the business. Marking up reviews you do not show, or services you do not offer, breaks Google’s guidelines and can lead to manual actions.

Structured data does not promise rich results or better rankings. What it does is make your facts unambiguous to every system that reads it, which includes Google, Bing and the search tools that sit behind AI assistants. For a local business, unambiguous facts are most of the battle.

Schema types a local business should have

Most local business sites need only a handful of schema types, used correctly and consistently. Start with LocalBusiness, or better, the most specific subtype that fits: Dentist, Plumber, RoofingContractor, HVACBusiness, Attorney, AutoRepair, Restaurant and many others exist on schema.org. Include the name, URL, logo, telephone, address or service area, opening hours, geo coordinates and sameAs links to your official profiles.

Add Organization data for the company as a whole, especially if you have several locations, and give each location its own LocalBusiness entry on its own page. Use Service markup on service pages to describe what the service is and the area it covers. Use FAQPage on pages that contain real question-and-answer sections, keeping in mind that Google now shows FAQ rich results only for a narrow set of authoritative sites, so the value is mostly in clarity rather than in a visual result.

Article or BlogPosting belongs on guides and blog posts, with a real author or organization and accurate publish and update dates. BreadcrumbList belongs on nearly every page. WebSite markup on the homepage identifies the site and its name. Review and AggregateRating markup should be used only where the rules allow, and Google does not show star ratings for a business’s reviews of itself on its own site.

  • LocalBusiness or a specific subtype such as Plumber or Dentist
  • Organization for the company, with sameAs profile links
  • Service on each service page
  • BreadcrumbList on every page below the homepage
  • Article or BlogPosting on guides and posts
  • FAQPage where a page has a real FAQ section
  • WebSite on the homepage

Breadcrumbs are the short trail near the top of a page that shows where you are: Home, then Services, then Drain Cleaning, then Drain Cleaning in a particular city. They help visitors move up a level without hitting the back button, and they tell search engines how the site is organized.

Google can show breadcrumb paths in search results in place of a raw URL, which makes a listing easier to read. More importantly, a clean hierarchy helps crawlers understand that your city pages belong under a service, and your services belong under the business. That structure is part of how a large site stays understandable as it grows.

Use visible breadcrumbs and matching BreadcrumbList structured data, and make sure each step links to a real page. A breadcrumb trail that points to a category page that does not exist, or that changes from page to page, sends mixed signals and frustrates visitors.

XML sitemaps

An XML sitemap is a list of the URLs you want search engines to know about. It does not force anything into the index, but it helps crawlers discover pages, especially new ones and deep ones that are a few clicks from the homepage. It lives at a predictable address, often yoursite.com/sitemap.xml, and you submit it in Google Search Console and Bing Webmaster Tools.

Keep it honest. A sitemap should list only pages that return a normal response, are meant to be indexed and point to themselves as the canonical version. Leaving in redirected URLs, broken links or pages marked noindex teaches search engines not to trust the file. The lastmod date should change only when the page content actually changes, not every time the site rebuilds.

Large sites can split their sitemaps by type, such as one for services, one for locations and one for guides, tied together by a sitemap index. That makes problems easier to spot in Search Console, since you can see which group of pages is being indexed and which is not.

robots.txt

robots.txt is a short text file at the root of your site that tells crawlers which parts they may visit. It is a set of requests that well-behaved crawlers follow. It is not security, and anything truly private should be protected by a login, not by a line in robots.txt.

For most local business sites, the right robots.txt is short. Allow everything public, disallow admin areas, form endpoints and internal search results, and include a line pointing to the sitemap. The most common serious mistake is a leftover Disallow rule from a staging site that blocks the whole live site. It happens more often than anyone likes to admit, usually right after a redesign.

robots.txt is also where you decide how to treat AI crawlers. Each company publishes the names of its crawlers, such as GPTBot and OAI-SearchBot for OpenAI, ClaudeBot for Anthropic and PerplexityBot for Perplexity. Google-Extended is a separate control that governs whether Google may use your content for training its Gemini models; it does not affect Google Search or AI Overviews. For a business that wants to be found and recommended, allowing the search crawlers is usually the right call.

How it all fits together

None of these files works alone. Structured data describes your facts. Breadcrumbs and internal links describe your structure. The sitemap lists your pages. robots.txt sets the rules of access. llms.txt offers a curated summary for tools that care to read it. The website itself, with clear, specific content, is what all of these point to.

On the sites we build, all of this is generated from the same source of truth as the pages themselves, so the hours in the schema always match the hours on the page and every new page lands in the sitemap automatically. That consistency matters more than any single tag. When every signal on and off your site agrees, search engines and AI systems can describe your business without hedging.

Want this done for you? Think SEO audits your site for free, sends a price in writing, and nothing is due until you reach page one of Google search. Get the free audit.

I can accept failure, everyone fails at something. But I can’t accept not trying.

Michael JordanSix-time NBA champion

Related questions

Do I need an llms.txt file?

You do not need one, but it is reasonable to have. It costs little, does no harm and gives AI tools that choose to read it a clean summary of your business and key pages. It will not by itself change your rankings or make AI assistants recommend you. Spend your effort on structured data, consistent listings and strong content first.

Will adding schema markup get me star ratings in Google?

Not usually for a local business’s own reviews. Google does not show review stars for a business that marks up reviews of itself on its own site. Schema still helps by stating your name, hours, location, services and profiles clearly, which supports accurate listings in search, in Google’s knowledge panel and in AI answers that draw on those facts.

What is the difference between robots.txt and a noindex tag?

robots.txt controls crawling: whether a bot may fetch a page. A noindex tag controls indexing: whether a fetched page may appear in search results. If you block a page in robots.txt, Google cannot see its noindex tag, so the URL can still appear in results without a description. To keep a page out of search, allow crawling and use noindex.

Which schema type should a service business use?

Use the most specific LocalBusiness subtype schema.org offers for your trade, such as Plumber, Electrician, RoofingContractor, HVACBusiness, Dentist or Attorney. If none fits, use LocalBusiness or ProfessionalService. Pair it with Service markup on each service page and BreadcrumbList across the site, and make sure every fact in the markup matches the visible page.

Start with a free audit.

We read your site the way Google does, check where you rank across your market and look at your Google Business Profile. You get what we found and a price in writing. We usually get there within a week of launch, and nothing is due until you reach page one of Google search.