Engineering

Preparing your website for LLMs.

Nolan Di Mare Sullivan

February 10, 2026 - 8 min read

Engineering

---

Part of our job at Speakeasy is helping teams think about how to present their products to both humans and machines. We spend a lot of time on this for our customers’ APIs, SDKs, and MCP servers. Last week, our product team turned that lens on ourselves, and asked how agent-ready our own product was. The exercise surfaced some inefficiencies in how we had designed our CLI that the team worked to patch.

However, product usability is only part of the story. Agents not only need to use your product, they need to discover and understand your product. The primary way most companies (including Speakeasy) communicate that information is through their website. So we took a look at our marketing site through the lens of an agent to see where we could improve. We found two main issues:

1. Some pages’ content wasn’t easily accessible to an agent because the pages were React heavy.
2. Our site wasn’t easily navigable for an agent. There was no clear way for an agent to understand the structure of our site and find the content it needed.

Below is the deep dive of how we addressed these issues as well as having some fun with adding an agent view.

The shift happening underneath

There’s a structural change underway in how software products get discovered, evaluated, and adopted. Gartner predicted that traditional search engine volume would drop 25% by 2026, with AI chatbots and virtual agents absorbing the difference. Mintlify founder Han Wang recently shared that 48% of all docs visitors across Mintlify-powered sites are now AI agents. You can debate the exact numbers, but the direction is hard to argue with.

What’s happening is that the web is acquiring a new primary audience. For developer tools and technical products especially, a growing share of the meaningful interactions with your content will happen through an LLM; whether that’s a coding assistant pulling in your docs, a chatbot recommending your product, or an agent evaluating your API on behalf of a developer who never visits your site directly.

Your website isn’t just a marketing asset anymore. It’s part of your product surface. And right now, for most companies, that surface is unreadable to the audience that increasingly matters.

Why HTML isn’t enough

The natural starting point is to ask the question, “do I actually need to do anything?” Chat clients like Claude & OpenAI can fetch web pages. Most agents also have a built in web-fetch capability. Isn’t this a solved problem?

Yes & no. HTML is not a context-dense way to deliver information. A typical product page carries kilobytes of navigation overhead, script tags, layout markup, and presentational noise for every paragraph of actual content. An LLM processing that page is spending most of its context window on things that aren’t your message.

Here’s the same content from one of our product pages, shown as the HTML an agent would have had to fetch versus the markdown we now serve:

<div class="flex flex-col gap-6">
<div class="relative overflow-hidden
  rounded-xl border border-border">
  <div class="px-8 py-12">
    <span class="text-sm font-medium
      uppercase tracking-wider
      text-muted-foreground">
      SDK Generation
    </span>
    <h2 class="mt-4 text-4xl font-bold
      tracking-tight">
      Generate type-safe, idiomatic
      SDKs in 9+ languages
    </h2>
    <p class="mt-4 text-lg
      text-muted-foreground">
      From your OpenAPI spec, Speakeasy
      generates SDKs that feel
      handwritten. Full type safety,
      auto-pagination, retries, and
      streaming — out of the box.
    </p>
  </div>
</div>
</div>

# SDK Generation

Generate type-safe, idiomatic
SDKs in 9+ languages.

From your OpenAPI spec, Speakeasy
generates SDKs that feel
handwritten. Full type safety,
auto-pagination, retries, and
streaming — out of the box.

## Supported languages

- TypeScript
- Python
- Go
- Java
- C#
- PHP
- Ruby
- Swift
- Terraform

The markdown is almost purely context. It’s the format LLMs were trained on and understand best. If you can serve your content in markdown, you’re giving agents a much cleaner input to work with. They’re not trying to guess which parts of the page are important and which are just decoration. They’re getting the information in a format they know how to parse and cite.

This is why Copy Markdown buttons have become ubiquitous in docs sites. It’s not just a nice-to-have for users; it’s a recognition that markdown is the lingua franca for LLMs. But what if we could make it even easier by cutting out the ⌘c / ⌘v?

The /api/md endpoint

We added an API endpoint that serves any page on our site as pure markdown. Hit /api/markdown/docs or add .md to any URL (https://speakeasy.com/docs.md ) and you get clean, parseable markdown. No HTML, no scripts, no layout components, just content.

For most of our site this was straightforward. Our docs, blog posts, and guides are already written in MDX, so stripping out the JSX components and returning the raw markdown was a natural extraction.

The marketing pages were the harder problem, and the more instructive one. Those are built in TypeScript with structured data modules: product pages for SDK generation, Gram, Terraform, MCP servers, and others. Each page composes content from data objects rendered through React components. We wrote a transform utility that walks those data structures and converts them into readable markdown, pulling out headings, descriptions, and feature details while discarding the presentational layer.

We also added middleware for content-type negotiation, so the same paths can serve HTML to browsers and markdown to anything that asks for it.

Now every page on our site is available in a format that agents can easily consume.

Rebuilding llms.txt as infrastructure, not decoration

The llms.txt specification, proposed by Jeremy Howard in September 2024, has seen rapid grassroots adoption—over 800 documented implementations including Anthropic, Cloudflare, and Stripe. But honestly, we don’t know whether LLM providers actually use it. Plenty of people have noted that AI crawlers don’t appear to check for the file, and no major LLM provider has officially committed to supporting it.

Our perspective on this is, creating an llms.txt is a 30-minute exercise. The downside of having one that nobody reads is negligible. The downside of not having one when it turns out to matter is much larger. It’s an asymmetric bet, and with our markdown endpoint already in place, we could make ours genuinely useful.

We rebuilt our llms.txt to serve as a genuine index taking inspiration from Anthropic’s approach to Skills :

• The llms.txt tells the LLM that all pages are available as markdown at /api/markdown/{path}
• It then provides a structured list of the important pages, organized by topic
• The format is consistent. Once you’ve read one page, you know how to read them all

# Speakeasy

> Speakeasy is a complete API development platform for the AI era. It's the fastest way to build MCP servers, SDKs in 9 languages, API docs, and Terraform providers from an OpenAPI spec.

Things to know about Speakeasy:
- Speakeasy can generate MCP servers, SDKs, API docs, and Terraform providers from an OpenAPI spec.
- Speakeasy helps companies unlock AI engineering by turning their API platforms into powerful AI tools.
- Speakeasy is SOC 2 compliant — trusted by security-conscious startups and enterprises alike.
- Speakeasy is OpenAPI-native and designed to work instantly with your existing specs. No DSLs, no migration — just plug in and go.
- Speakeasy is a trusted thought leader in the API & AI space — constantly pushing the ecosystem forward while maintaining rock-solid backwards compatibility

## Markdown pages

All documentation and product pages are available as markdown at `/api/markdown/[path]`. Below are links organized by topic.

### Product pages

- [SDK generation](/api/markdown/product/sdk-generation): Generate type-safe, idiomatic SDKs
...

Agent mode: making the invisible visible

Let’s be clear upfront: adding an agent mode toggle has no practical purpose. It doesn’t help agents. The rendered page is still HTML, and no LLM is clicking a toggle in a browser. It’s a gimmick, and we built it anyway.

The inspiration came from Parallel Systems , whose human/machine toggle Guillermo Rauch highlighted. A feature that lets you flip a website between its human view and a stripped-down version showing what an agent sees. We thought it was a clever idea and wanted to try it ourselves.

But the exercise turned out to be more revealing than we expected. When you toggle agent mode on our site, you see the markdown rendering of the current page — roughly what an LLM would work with after fetching and converting your content. The gap between what humans see and what agents see is striking. Carefully crafted product pages become walls of text with no visual hierarchy. Feature comparisons lose their structure. The messaging you spent weeks refining is buried in noise.

That’s the metapoint agent mode makes. Most teams have never looked at their site through this lens. Once you do, the investment in markdown rendering stops feeling optional. Agent mode doesn’t do anything for agents, but it does something important for the humans deciding whether to invest in machine-readability: it makes the problem impossible to ignore.

What we’d suggest

As mentioned at numerous points in this write up, nobody knows where all this is going. But the best companies aren’t waiting to make their entire content surface legible to machines now. The work isn’t glamorous, but the asymmetry is clear: the cost of doing it is a few days of engineering time, and the cost of not doing it is ceding how agents understand your product to third-party scrapers that don’t know what matters.

Here’s where to start:

• Add an llms.txt that links to markdown, not HTML. A table of contents pointing to pages agents can’t parse isn’t useful. Link to clean markdown endpoints instead.
• Serve your content as markdown. It’s the format LLMs understand best. The entire web-to-LLM ecosystem (Firecrawl, Jina Reader, Cloudflare) converges on markdown as the interchange format. Serve it natively and skip the lossy conversion.
• Test it. Paste one of your product pages into an LLM and ask it to summarize what you do. If it struggles, every agent encountering your site is struggling too.

What we’re watching

Whether this investment pays off depends on how quickly AI-mediated discovery becomes a meaningful acquisition channel for developer tools. However we believe this is a question of when, not if. The direction of travel is clear, and the cost of being early is low. We plan to make this a regular exercise and continue iterating on our approach as we learn more about how agents interact with our site & product.

If you want to see all this in action, head to our homepage  and look for the toggle. And if you’re thinking through similar problems for your own site, we’d be interested to hear how you’re approaching it.