What Does an AI Engine See When Your Page Needs JavaScript?

What does an AI engine see when your page needs JavaScript?

Nobody outside those companies can tell you for certain, and that is the honest answer. Google documents its rendering behaviour in detail. The AI crawler documentation does not address JavaScript at all. So the safe assumption is that anything only a browser can assemble may never reach the model.

This question comes up on almost every technical call I take now. A founder has a site where the pricing table, the FAQ, or the entire blog index only exists after JavaScript runs, and they want to know whether ChatGPT and Perplexity can read it.

I can give you exactly what the vendors document, which is more than most articles on this topic will do, and then I can tell you what I actually do about it.

How does Google handle JavaScript today?

In three phases that Google names plainly: crawling, rendering, indexing. According to Google's JavaScript SEO documentation, Googlebot queues all pages with a 200 HTTP status code for rendering, unless a robots meta tag or header tells Google not to index the page. Rendering happens separately from crawling.

The rendering step is done by a real browser. Google's documentation says that once Google's resources allow, a headless Chromium renders the page and executes the JavaScript. That is not a simplified fetch or a partial parse. It is Chromium running your code.

But notice the conditional. Once Google's resources allow. The same documentation says a page may stay on the rendering queue for a few seconds, but that it can take longer than that. Rendering is a budgeted service, not a guarantee delivered on your schedule.

Two more details from that page are worth committing to memory. Google Search will not render JavaScript from blocked files or on blocked pages, so a robots.txt rule that blocks your script directory blocks your content. And Google can only discover your links if they are anchor elements with an href attribute, though Google explicitly says it is fine to use JavaScript to inject links into the DOM as long as those links follow the best practices for crawlable links.

What do the AI crawlers actually tell you about rendering?

Nothing. I fetched OpenAI's own crawler documentation while writing this, and it names four crawlers with their user agents and purposes, but it contains no statement at all about whether any of them executes or renders JavaScript. That silence is the single most important fact in this article.

Here is what the page does say. OAI-SearchBot is used to surface websites in search results in ChatGPT's search features. GPTBot is used to crawl content that may be used in training OpenAI's generative AI foundation models. OAI-AdsBot validates the safety of ad landing pages, and OpenAI states that the data collected by OAI-AdsBot is not used to train generative AI foundation models.

The fourth is the interesting one. ChatGPT-User covers user-initiated actions, and OpenAI's own wording is that because these actions are initiated by a user, robots.txt rules may not apply. That is a meaningful distinction between a crawler harvesting the web and an agent fetching a page because a person asked it to.

One operational detail from the same page: OpenAI says it can take around twenty four hours from a site's robots.txt update for its systems to adjust. If you are changing access rules, do not expect the change to land the same afternoon.

Why does that documentation gap matter so much?

Because the internet is full of confident claims in both directions, and none of them are sourced. I have read that AI crawlers never run JavaScript and I have read that they all do now. Neither claim appears in the vendor documentation, which means anyone asserting it is guessing or repeating someone else's guess.

My position is that unverified in this situation should be treated as unavailable. You are not choosing between certainty and uncertainty. You are choosing between a page whose content is guaranteed to be in the HTML and a page whose content might be, depending on undocumented behaviour that can change without an announcement.

That is not a hard call. The cost of putting your important content in the initial HTML response is low. The cost of finding out in six months that it was never readable is high and completely invisible while it is happening, which is the worst property a problem can have.

Which parts of a page go missing most often?

The parts built as interactive widgets. Tabbed feature comparisons, accordion FAQs whose answers load on click, pricing tables driven by a currency or billing toggle, review carousels pulled from a third-party script, and blog indexes that paginate through an API call rather than real links.

Accordions deserve a special mention because they are so common and the failure is so subtle. If the answer text is present in the HTML and merely hidden by CSS, you are fine. If the answer is fetched when the user clicks, the answer does not exist until a human interacts with the page, and no crawler is clicking anything.

Third-party review widgets are the next most common casualty. The stars and the testimonial text live in an iframe or get injected by a vendor script, which means your strongest trust content is the content least likely to be read. That is close to the problem I described in gating content that AI engines cannot read, except nobody chose to gate it.

The one that costs the most is the paginated blog index. If your archive is only reachable by clicking a load more button that fires an API request, a crawler may see your first twelve posts and nothing else. Everything you published before that point is orphaned in practice, whatever your sitemap says.

How do you check what a crawler actually sees?

Disable JavaScript in your browser and reload the page. Whatever remains is roughly your floor. Then view the raw HTML source, not the inspector, because the inspector shows you the DOM after scripts have run and will quietly reassure you about content that was never in the response.

The difference between view source and the element inspector is the single most useful thing I teach people on these calls. Right-click and choose view page source, then use find to search for a sentence from your most important paragraph. If the sentence is not there, it arrived via JavaScript.

Do this on four page types rather than one: your homepage, your pricing page, one blog post, and your blog index. Those four cover most of the ways a site fails. If you want a more systematic version, the same check belongs in a recurring audit alongside whatever you already do for llms.txt and AI crawler access.

One caution. A page can pass the view source test and still be invisible if your robots.txt blocks the resources or the path. Check access rules and rendering separately, because they fail independently and produce identical symptoms.

Does this mean you should abandon JavaScript?

No, and anyone telling you that is selling simplicity you cannot afford. JavaScript is how modern interfaces work. The rule is narrower than that: the content you want quoted, cited, or indexed should be in the HTML response, and JavaScript should enhance what is already there rather than supply it.

That principle is old enough to have a name, progressive enhancement, and it survived every platform shift since because it is fundamentally about not making your content contingent on code executing correctly. Build the interaction on top of real text and real anchor elements. Let the script improve the experience for people who have a browser.

In practice this means choosing the server-rendered option when your framework offers one, keeping FAQ answers in the markup and using script only to show and hide them, and making sure every navigational link is a genuine anchor with an href rather than a div with a click handler. Google says that last part outright, and it is the cheapest rule on the list to follow.

What does this mean for a site built in a visual builder?

Check rather than assume. Open one of your own published pages, view the raw source, and search for a sentence from your main content. Do the same for your blog index. Whatever the platform's marketing says about performance and SEO, the source of your actual page is the only evidence that settles it.

I do this on every site I take over, including Webflow sites, and I recommend you do it even if you are confident. It takes ninety seconds per page type. The habit is worth more than any general claim I could make about a platform, because platforms change and your specific build might use a custom embed or an integration that behaves differently from the default.

Pay particular attention to anything you added yourself. Custom code embeds, third-party scripts, and CMS filtering widgets are where the risk concentrates, because those are the parts the platform did not generate and is not responsible for. If you added an animated stats counter that starts at zero and counts up, congratulations, your best numbers are the number zero as far as anything reading the HTML is concerned.

What should you do next?

Pick your four most important pages. View the raw source of each and search for a sentence you would want an AI engine to quote back to a buyer. Anything missing is your work queue, ordered by how badly you want that sentence read. Fix the top one this week.

Then stop trying to reverse-engineer undocumented crawler behaviour. It is a genuinely interesting puzzle and it is also a trap, because the answer changes without notice and you cannot verify it. Put the content in the HTML, and the question stops mattering. That is also why I keep coming back to how AI search picks the paragraph it quotes, because once the text is reachable, the next problem is whether it is worth quoting.

If you want someone to run this check across your site and tell you what is actually missing rather than what might be, reach out. It is a short piece of work with an unusually clear answer at the end. Let's chat.