The customer who used to type „metal fabrication near me“ into Google and click through three websites now, some of the time, asks an assistant and gets an answer. That answer names two or three companies. You are either in it or you are not, and either way nobody clicks through to your site.
It sounds like a problem for big brands. It is not. Language models answer questions like „who does electrical inspections around here“ too, and they build those answers from content their crawlers collected earlier. Your company is in that content, or it is not.
This article shows who actually visits your website (and why your analytics does not show it), what a model takes away from a page, what is worth doing, and what is marketing folklore.
Short answer
- Language model crawlers already visit ordinary company websites. In our own measurement they made more requests in a week than Googlebot.
- You will not see them in Google Analytics. Analytics is sent from the browser and most of these crawlers do not run JavaScript. They only show up in server logs.
- Models read text, headings, tables and questions with answers. They cannot take a figure out of an image, a carousel, or content a script fills in after loading.
- The biggest single lever is not technical. It is having direct answers to the questions people actually ask, written so they can be quoted without editing.
- The llms.txt file is a proposal no major model has committed to. Anyone selling it as the answer is selling you hope.
What actually changed
For twenty years search worked by showing ten links and letting you choose. Whoever came third got a click. Now there is a layer that replaces links with an answer: Google's overview above the results, an answer in ChatGPT, an answer in Perplexity, the assistant on a phone.
For a company that changes two things. First, some people never reach your site at all, because they got the answer sooner. Second, and this matters more, you can be in that answer even when you are not on the first page of results. Models do not take the top three results; they take content they can read and that answers the question clearly.
For a smaller company that is better news than bad. A competitor with high rankings built on years of link investment does not carry all of that advantage across into a model's answer.
Who visits your site without you knowing
These are not figures from a slide deck. We pulled them out of our own server logs over seven days on one of our sites, and they are what they are.
| Crawler | Whose it is | Requests | Distinct pages |
|---|---|---|---|
| Bytespider | ByteDance, that is TikTok | 191 | 42 |
| Meta | Facebook and Instagram | 62 | 62 |
| Googlebot | Google, classic search | 39 | 19 |
| GPTBot | OpenAI, content collection | 26 | 19 |
| Bingbot | Microsoft | 21 | 7 |
| Applebot | Apple | 14 | 14 |
| OAI-SearchBot | OpenAI, search inside ChatGPT | 11 | 1 |
| SeznamBot | Seznam, the Czech search engine | 7 | 4 |
| ClaudeBot | Anthropic | 4 | 2 |
It is one site over one week, so let us not turn it into a law. As a picture of what is going on it is enough: traffic from language model crawlers is no longer marginal, and taken together it matches Google's.
The last column is the more interesting one. GPTBot went through nineteen different pages, so it was collecting content. OAI-SearchBot hit one single address eleven times, which looks like checking one specific thing. These crawlers do different jobs and it is worth not lumping them together.
Why your own analytics does not show this
This is the finding that makes the rest of the article worth reading, and we only found it by comparing the two sources.
Our own analytics module caught one of all those crawlers. Not because it is bad, but because it works like every other one: a piece of code on the page runs in the browser and reports a page view. A crawler that downloads the page and reads it runs no code at all. So it never reaches the analytics.
Exactly the same applies to Google Analytics and to every tool built on a tracking script. Looking at your statistics and seeing no AI crawlers does not mean they are not there. It means you are looking through a tool that cannot see them.
A browser tracking script (Google Analytics and the rest): sees people, and the crawlers that run JavaScript. Most AI crawlers are invisible to it.
Server logs: see every request that reached the site, including crawlers that only download text. They are dull and nobody ever looks at them.
What a model takes from a page, and what it misses
A model does not read a website the way a person does. It does not see colour, it has no sense of layout and it does not care about animation. It takes text and its structure. Which has some uncomfortable consequences for sites that look modern.
| Content on the page | Gets read | Why |
|---|---|---|
| Text in paragraphs and headings | yes | The most readable form there is |
| A table with a header row | yes | The relationship between figures is legible |
| A question with the answer under it | yes | Can be quoted without editing |
| A price list as text | yes | The number can be extracted |
| A price list as an image or scanned PDF | no | An image is just a smudge to a reader |
| Figures inside a carousel or tabs | often not | They appear only after a click |
| Content drawn in by a script after loading | often not | The crawler will not wait for it |
| Contact details only as an image in the footer | no | The most common mistake on small sites |
The most common failing is not technical, it is a writing problem. A company writes three paragraphs on a service page about how long it has been trading and how much it values quality, and never says what it actually does, for whom, where and for how much. A model has nothing to quote, because there is no claim there.
Try to find one sentence on your own service page that could be lifted out and used as an answer to a customer's question. If there is no such sentence, that is a job for a copywriter, not a developer.
If not one sentence can be lifted from your service page to answer a question, there is nothing to quote
Eight steps worth taking
In order of value against effort. A company can do the first three itself in an afternoon.
- 1Write down twenty questions customers actually ask you, awkward ones about price and lead times included. Do not invent them, pull them out of e-mails and phone calls.
- 2Answer each in three sentences, concretely and with a number. „We inspect a distribution board within five working days, from 2 500 CZK“ is an answer. „We offer professional inspection services“ is not.
- 3Put those questions on the site as text, not inside an accordion loaded by a script. The best place is at the end of the page about that service.
- 4Check that your prices, contact details and address can be read as text. Not as an image, not as a PDF, not only after a click.
- 5Make the company details identical everywhere they appear: website, Google Business Profile, directories, social profiles. Conflicting addresses and phone numbers reduce confidence in all versions.
- 6Have structured data added for the company and for the questions and answers. It is a machine readable description of what the page says, and it is hours of work, not days.
- 7Confirm the page can be read without running scripts. The quickest test: view the page source in your browser and search it for text you can see on the page. If it is not there, a crawler will not see it either.
- 8Ask the models about your company and write down what they say. Repeat in two months. It is the only measurement in this area that genuinely works.
A test you can run in ten minutes
Open two or three different assistants and ask them five questions. Not „what is company XY“, which is a weak test. Ask as a customer who has never heard of you:
- Who does your service in your area
- What does your service cost
- What to watch out for when choosing a supplier of your service
- Which companies supply your product for your industry
- Your company name directly, and what the model knows about it
Write the answers into a table with the date. Three things matter: whether you get mentioned, whether what it says about you is true, and where it got that from. The last one is usually the most useful: often it turns out the model is drawing on a directory or an article somewhere else rather than on your site. Correcting the details there pays off before touching your own website.
What does not work, however well it sells
This field has produced more promises than results in the past year. Three we run into most.
The llms.txt file. A proposal for telling models in machine readable form what matters on a site. It sounds sensible and takes ten minutes. But no major model has yet confirmed using it. Make one by all means, it will do no harm, but anyone selling it as a service for real money is selling hope.
Publishing articles in bulk for the models. Fifty keyword generated texts a month will not raise how often you are cited, because a model is not looking for volume, it is looking for a claim it can use. Ten pages with concrete answers will do more than a hundred pages of general content.
Promises of guaranteed placement in answers. A model's answer is generated afresh each time and varies with the wording of the question and with who is asking. Placement cannot be guaranteed, any more than first place in Google ever could.
What about just blocking the crawlers
A fair question with no clean answer. Crawlers can be blocked by name in robots.txt, so you can say: collect my content, or keep off it.
For a company that wants to be found, blocking makes little sense: the content is there to be read. For a publisher whose living is the content, it can make sense, because the model uses the work and sends no reader back. That is an honest dispute and everyone has to settle it for themselves.
One thing gets confused constantly though: blocking collection is not the same as blocking appearance in an answer. When a model builds an answer from search it may still point at you even though you blocked content collection. And the reverse: blocking can shut you out of answers you would otherwise have been in.
How we handle it
Every package in our price list carries a line about the groundwork for Google and for AI search. Here is what actually sits behind it.
- Pages are rendered on the server, so a crawler receives finished text rather than an empty page for a script to fill in.
- Structured data for the company, articles and questions is part of the site, not an extra. This article carries it too, including the six questions at the end.
- Questions and answers are a block that can go on any page, not text hidden inside a collapsible widget.
- We measure crawlers by name. Our analytics distinguishes them, and whatever it misses we look up in the server logs. That is exactly how the table above came about.
- Zero third party scripts on a page means there is nothing for the content to get stuck behind.
And the honest other side: nobody, us included, can promise a model will cite you. What can be done is making sure there is something to cite and that it can be read. The rest depends on how you are written about elsewhere, which is marketing work rather than website work.
Frequently asked questions
How do I get my business cited by ChatGPT and Google AI Overviews?
Put direct answers to the questions your customers actually ask on your website, written as text with concrete numbers, and have them rendered on the server so a crawler receives them finished. Then make your company details identical everywhere and add structured data. Placement in an answer cannot be guaranteed, any more than first place in Google ever could.
Why can I not see AI crawlers in my analytics?
Because Google Analytics and similar tools measure with code that runs in the browser, and most language model crawlers do not run JavaScript. They only show up in server logs. So an empty figure in your analytics does not mean the crawlers are not visiting, it means you are looking through a tool that cannot see them.
Do AI crawlers visit small company websites?
Yes. On one of our sites, OpenAI and Anthropic crawlers together made 41 requests over seven days against 39 from Googlebot. That is one site over one week, so it is not a rule, but it is no longer a marginal phenomenon either.
Does llms.txt work?
It is a proposal that no major model has publicly committed to using. Creating one takes minutes and does no harm, but as a paid service it is selling hope. What demonstrably helps is something else: readable text with concrete answers on the pages themselves.
Should I block AI crawlers from my site?
For a company that wants to be found it makes little sense: the content is there to be read. For a publisher whose living is the content it can. Watch out for a common misunderstanding: blocking content collection is not the same as blocking appearance in an answer, and blocking can shut you out of answers you would otherwise have been in.
How do I check whether models know about my company?
Ask them. Not about your company name, but as a customer who has never heard of you: who does your service in your area, what it costs, what to watch out for when choosing a supplier. Write the answers down with the date and repeat in two months. What matters is whether you are mentioned, whether it is true, and where it came from.
Summary
- Language model crawlers already visit ordinary company websites. Over our measured week they matched Googlebot.
- You will not see them in your analytics, because a browser tracking script cannot catch them by design. Server logs can.
- Models take text and its structure. A price list as an image, contact details as a footer graphic and script rendered content do not exist for them.
- The biggest lever is a writing one, not a technical one: concrete answers with numbers instead of paragraphs about quality and tradition.
- llms.txt is a proposal with no confirmed consumer. Bulk generated articles will not raise how often you are cited.
- Blocking crawlers makes sense for publishers, not for a company that wants to be found. And blocking collection is not blocking appearance.
Want to know whether the crawlers can read you
Send us your website address. We will check whether the content can be read without running scripts, whether the site carries structured data, and whether anything on it can be quoted at all. We will send it back in plain language, including what you could fix yourself.