Request // Response

Request // Response Episode 3: Anthropic MCP, GraphQL vs REST, and LLM API strategies

Sagar Batchu

April 14, 2025

Show Notes:

• [00:01:08] What OpsLevel does and why internal developer portals matter
• [00:01:51] Ken’s journey from PagerDuty to Shopify and to starting OpsLevel
• [00:04:03] Developer empathy, platform engineering, and founding OpsLevel
• [00:05:02] OpsLevel’s API-first approach and extensibility focus
• [00:06:26] Using GraphQL at scale—Shopify’s journey and lessons
• [00:08:30] Managing GraphQL performance and versioning
• [00:10:12] GraphQL vs REST – developer expectations and real-world tradeoffs
• [00:11:27] Key metrics to track in API platforms
• [00:12:50] Why not every feature should have an API—and how to decide
• [00:13:48] Advice for API teams launching today
• [00:14:44] API consistency and avoiding Conway’s Law
• [00:15:50] Agentic APIs, LLMs, and the challenge of semantic understanding
• [00:18:48] Why LLM-driven API interaction isn’t quite magic—yet
• [00:20:00] Internal APIs as the next frontier for LLM productivity
• [00:21:43] 5x–10x improvements in DX via LLMs + internal API visibility
• [00:23:00] API consolidation, discoverability, and LLM-powered developer tools
• [00:23:54] What great developer experience (DevEx) actually looks like

More Quotes From The Discussion

Challenges of Running GraphQL

“I can tell you as OpsLevel, running a GraphQL API, you know, all the usual things, making sure that like you don’t have n+1 queries, ensuring, you don’t get per resource rate limiting like you do with REST… You have to kind of be more intentional about things like query complexity. Those are challenges you end up having to solve.

Versioning is another big one. We are far enough along in our journey. We haven’t done a major version bump yet. I know a few years ago, after I left Shopify, they switched their GraphQL schema to be versioned and now they have much more kind of program management around like every six months they have a new major version bump.

They require clients to migrate to the latest version, but that’s effort and that’s calories that you have to spend in that kind of API program management.”

DevEx of REST vs GraphQL

“We have a customer that has been with us for years, but our champion there hates GraphQL. Like he really hates it. And the first time he told me that I actually thought he was like, you know, joking or being sarcastic.

No. He was legitimate and serious because from his perspective, he has a certain workflow he’s trying to accomplish, like a “Guys, I just need a list of services. Why can’t I just like REST, even make a REST call and fetch last services and that’s it. Why do I have to do all this stuff and build up a query and pass in this particular print?”

And I get that frustration for developers that, you know, REST is sort of easier to start. It’s easier just to create a curl request and be done with it, right? It’s easier to pipe, the output of a REST call, which is generally just a nice JSON up into whatever you want versus “No, you’ve actually have to define the schema and things change.”

I think GraphQL solves a certain set of problems, again, around over fetching and if you have a variety of different clients. But there is this attractive part of REST, which is “I just made a single API call the single endpoint to get the one thing I needed, and that was it.” And if your use case is that, then the complexity of GraphQL can really be overwhelming.”

API Consistency and Conway’s Law

“I do think consistency is an important thing, especially when you’re dealing with a large company that has disparate parts working on different aspects of the API. You don’t want Conway’s Law to appear in your API—you know, where you can see the organizational structure reflected in how the API is shipped.

So making sure that an API platform team or someone is providing guidance on how your organization thinks about the shape of APIs is crucial. Here’s how you should structure requests. Here’s how you should name parameters. Here’s what response formats should look like. Here’s consistent context for returns and responses.

Here’s how to implement pagination. It’s all about ensuring consistency because the most frustrating thing as an API client is when you hit one endpoint and it works a certain way, then you hit another endpoint and wonder, ‘Why is this structured differently? Why does this feel different?’ That can be tremendously difficult. So having some guardrails or mechanisms to ensure consistency across an API surface area is really valuable.”

Referenced

• OpsLevel (opens in a new tab)
• Shopify Storefront API (opens in a new tab)
• GraphQL (opens in a new tab)
• REST (opens in a new tab)
• Conway’s Law (opens in a new tab)
• RED Metrics (opens in a new tab)
• Anthropic MCP (opens in a new tab)
• Agents.js (opens in a new tab)

Production by Shapeshift (opens in a new tab).

For inquiries about guesting on Request // Response, email samantha.wen@speakeasy.com.

Transcript

Introduction

[00:00:05] Nolan: Hey guys, this is Nolan. I do developer relations at Speakeasy. Thanks so much for taking the time to listen to the podcast. This is a new format for us and we’re actively looking to get feedback from the community. Would love if you guys could drop us an email, you can email me, nolan@speakeasy.com, or our producer Ira, ira@speakeasy.com.

[00:00:23] If you’re enjoying the content, don’t forget to sign up below, drop us your email, and you’ll be the first to hear about new guests that are coming on. Get episodes straight to your inbox as soon as they release. Thanks so much for listening and enjoy the rest of the episode.

Welcome and OpsLevel Introduction

[00:00:34] Sagar Batchu: Hey everyone. Welcome to another episode of Request and Response. I’m your host, Sagar, co-founder, CEO of Speakeasy. I’m joined today with Ken Rose, CTO, and co-founder of OpsLevel. Ken, how are you doing today?

[00:00:47] Ken Rose: I am doing great. Thank you. Thanks for having me on the show.

[00:00:49] Sagar Batchu: Absolutely. I hear you’re a pro at doing podcasts, so really I think we should swap places here.

[00:00:55] Ken Rose: It’s definitely new for me to be in the guest seat. It’s been a while. Normally I’m the host, but yeah, it’s fun to be on this side of the aisle today.

[00:01:01] Sagar Batchu: That’s awesome. Yeah. Cool. To kick things off, I would love for you to share a bit more about OpsLevel and what you guys do.

[00:01:08] Ken Rose: Yeah, definitely. So at OpsLevel, we make a product called an internal developer portal. And really this is a tool that helps engineering teams manage the complexity of really large software environments. And we help consolidate a bunch of the context from various tools that sort of exist out there across the building of software, the deployment of software, the management of software and production into a single, consolidated place so that engineering teams can ship more reliably and more securely.

The Journey to OpsLevel

[00:01:35] Sagar Batchu: That’s amazing. It honestly sounds like a product I know a lot of our customers would be interested in. I’ve worked at a few companies where building out like this single pane of glass is such a tough thing to do at scale. So it sounds like you’re solving a real problem. How did you come to working on this problem? What’s your journey to OpsLevel?

[00:01:51] Ken Rose: Yeah, it actually begins, oh man, like more than 12 years ago now. My co-founder, John, was the first employee at PagerDuty, right? The kind of, you know, big incident management company. And I was considering being their first Canadian employee. I was, I’m based here in Toronto, and at the time, again, it’s 12 years ago now. It was like, actually I wanna go start a startup. I really wanted to start my own company and do my own thing.

And I tried something in like real estate and it didn’t really go anywhere. And then I was like, you know, kind of went back from my tail between my legs “yeah, John, is that job offer still there?”

[00:02:19] I thought, you know, go back to PagerDuty. I worked for a year and then try again. Oh, that one year turned to four and John and I became really, you know, good friends and yeah, kind of like that seed was always there. And so I ended up going to another company after PagerDuty. It’s a small e-commerce company here in Canada called Shopify.

[00:02:35] And then I actually was on a parental leave. My daughter was born at the time and I kind of had that space to think about, what do I want in life? You know, what’s next? And that entrepreneurship bug was kind of still there. And so John also had a similar kind of fire brewing in his belly, and he reached out and we started, you know, getting to talking.

[00:02:52] And we had the, you know, the kind of list of all the possible startup ideas where a big list of 50 things, but whittled it down to two. And so one of them was a better applicant tracking system. So John and I had both done as engineering leaders, just a lot of interviews with engineering candidates.

[00:03:05] I won’t name which tool we use, but it wasn’t very good. It’s still around, so I don’t wanna name them. But we’re like, okay, we can build something better here. But in doing a lot of discovery, we realized the buyer for that is gonna be HR people. And I love my HR VP, but I wasn’t sure if I wanted to commit 10 years to working with that person or that persona.

[00:03:21] And the other idea was something in and around OpsLevel, and we had seen this idea of cataloging your services, building internal tooling to help development teams move more efficiently. We had seen that problem being solved internally as an internal tool at just a variety of organizations. We talked to Spotify before they’d open source Backstage, Shopify had their own version, LinkedIn, Microsoft, Groupon, like all these companies had basically reinvented the wheel, solving the same problem.

[00:03:45] And the thing we also love was the personas just it’s other developers, it’s other folks that were like John and myself. It’s developers, it’s platform engineering leaders. It’s engineering leadership at large. That’s my jam. I just, I love those folks, right? I, that’s the area I live in. The amount of empathy I have for that, the set of problems that those groups have is huge.

[00:04:03] So that was just, it was a very natural pull. And you know, we’re really fortunate years later, our, you know, company’s still thriving, growing, adding on new customers, and just really lucky to be able to kind of found lightning in a bottle and being able to build this great product for all of our customers.

APIs and OpsLevel

[00:04:16] Sagar Batchu: That’s amazing. I can resonate with so much of that story to be honest, that platform engineering is something I nerd out on so much. It honestly is that core part of many tech businesses today that is kind of silent hero in the background, giving the rest of the company velocity and the ability to create amazing products.

[00:04:35] The internal API and just more overall problem that OpsLevel is solving is not too far from actually why we started Speakeasy and wanting to kind of go after and bring consistency and governance and really, you know, just sanity to API development. I honestly feel like OpsLevel is, yeah would be something I could have used at any of the past companies I worked at, so that’s really amazing.

[00:04:57] Does OpsLevel have an API or how do APIs kind of fit into the OpsLevel story?

[00:05:02] Ken Rose: Yeah, no we have a large API, in fact so before starting OpsLevel years ago, again, when I was at PagerDuty, I was responsible, I as the sort of technical lead for their API. And then when I moved over to Shopify, I was the lead for their GraphQL API. Shopify has, as you know, massive API. Also, they had this API patterns team that was responsible for shepherding a lot of the shape of the API and was responsible for those kind of cross-cutting API concerns.

[00:05:25] For our API, we are very much like an API driven company. If you think about what goes into an internal developer portal, there’s an aspect of a software catalog. There is aspects of standardization, there’s aspects of developer self-service. There’s a ton of automation that has to go into driving each of those pillars.

[00:05:44] And the only way to achieve that automation is through an API. So pretty much everything we build as part of our platform strategy, we need to be API first because we really wanna be able to solve a large percentage of our customer’s problems, you know, as a product, but there’s always gonna be some long tail that we won’t be able to solve directly, right?

[00:06:01] Our scrappy team of product dev folks. And so for that, we need our APIs to be allowing for extensibility allowing for customization so that our customers can self-serve themselves. So, if there’s some tool we don’t support, if there is some particular workflow that like, makes sense maybe for one customer, but not for our base at large, we wanna make sure that our APIs are present and there to be able to do that.

[00:06:20] So that means that pretty much everything, and it’s a very broad product suite that you can do through the UI. There has to be an API for as well.

GraphQL vs REST: The Shopify Experience

[00:06:26] Sagar Batchu: That’s fantastic. Going back to really cool that you worked at Shopify during, I think such an amazing time there. One thing that stands out to me as an API developer, Shopify always got a lot of attention for being a company that actually used the GraphQL API externally with their customers.

[00:06:42] And I thought it was really cool to see that hot new technology actually go to scale and be used publicly. I feel like since then maybe this is a hot take, but since then, kind of the popularity on GraphQL, especially for external use cases, has gone up and down. There’s been a lot of debate around whether, you know, REST or GraphQL is the ultimate option.

[00:07:02] I know there’s no true answer, but I’m curious if that ever came up at Shopify. Did your customers, were they ever surprised that they had to integrate with GraphQL?

[00:07:10] Ken Rose: So I mean, I can talk a little bit and, you know, there’s history there, right? So I’m trying to remember this, like 2017, 2018, right? So I believe there was the marketplace API, and then there was the admin API. So I actually launched the admin API when I was there, right? It was Shopify has their marketplace API which was it marketplace or storefront? It might’ve been storefront, but it was like a smaller subset, APIs.

And that was almost like the test bed for GraphQL at Shopify. And that proved valuable for, you know, things like all the usual benefits that GraphQL provides, right? Because GraphQL kind of came out in 2015 from Facebook. But you know, ensuring that there’s no over fetching, ensuring better client experience.

[00:07:42] And so then there was this decision that they should migrate to their admin API, which is like the commerce API onto GraphQL. And the reasoning at a very high level is look, commerce is a graph. We had a very similar thesis when we started OpsLevel. We have a GraphQL API internally, because software development is a graph.

[00:07:59] There’s teams, there’s microservices, there’s applications, there’s this tapestry of stuff that’s kind of all connected, right? So GraphQL kind of makes sense there. Back to whether or not developers were surprised, I mean I don’t think anyone was surprised. ‘cause you know, there’s kind of enough lead up to the fact like, look, there was already this one API that was, you know, sort of using GraphQL.

[00:08:17] There was a lot of support. Shopify maintained backwards compatibility with their existing REST APIs for, you know, some period of time. So there was an overlap period to allow for that transition. Yeah, so, you know, I think overall it was like for Shopify and net positive, but it wasn’t without a lot of the usual hiccups that people kind of encounter when they’re dealing with GraphQL, right? Both as a client and as a company that provides it.

GraphQL Implementation Challenges

[00:08:30] On our side, I can tell you as OpsLevel, running a GraphQL API, you know, all the usual things, making sure that like you don’t have N plus one queries, ensuring, you know, you don’t get per resource rate limiting like you do with REST, you have to kind of be more intentional about things like query complexity. Those are, you know, challenges you have to end up having to solve. Versioning is another big one. You know, we are far enough along in our journey. We haven’t done a major version bump yet.

[00:08:58] I know a few years ago, this was after I left Shopify based switched their GraphQL schema to be versioned and now they have much more kind of program management around like every, I think it’s every six months they have a new major version bump. They require clients to migrate to the latest version, but that’s effort and that’s calories that you have to spend in that kind of API program management.

[00:09:15] Sagar Batchu: Yeah, that makes sense. I think that’s really where the argument kind of goes astray. A lot of the times I think is people you know, compare GraphQL to REST and then immediately look at how they’re used externally, but they don’t realize the kind of practices that go behind running these two different kinds of APIs is vastly different.

[00:09:33] And I think you raised a really good point, like commerce is a graph and that shows like kind of your choice on GraphQL was rooted in a use case that made sense. And I think it always reminds us as developers like not to debate these things in the abstract. That is kind of a key use case behind why you might choose GraphQL, and I think Shopify just happened to be one of the first ones at scale with a very, very clear use case for it.

[00:09:57] Ken Rose: Yeah, it’s funny. I agree with everything you just said and nevertheless, we have, you know, some customers, we deal with developers, right? And sometimes developers are prickly is, is maybe a way I would say describe it. So I know there’s one customer I think about in particular, I love them. They’re, a fantastic customer.

[00:10:12] They’ve been a customer of us for years, but our champion there is he hates GraphQL. Like he really hates it. And the first time he told me that I actually thought he was like, you know, joking or being sarcastic. No he was legitimate and serious because from his perspective, like he, you know, he has a certain workflow he’s trying to accomplish, like a “Guys, I just need a list of services. Why can’t I just like REST, even make a REST call and fetch last services and that’s it. Why do I have to do all this stuff and build up a query and pass in this particular print?”

And I get that frustration for developers that, you know, REST is sort of easier to start. It’s easier just to create a curl request and be done with it, right? It’s easier to pipe, the output of a REST call, which is generally just a nice JSON up into whatever you want versus “No, you’ve actually have to define the schema and things change.”

I think GraphQL solves a certain set of problems, again, around over fetching and if you have a variety of different clients. But for, you know, there is this attractive part of REST, which is “I just made a single API call the single endpoint to get the one thing I needed, and that was it.” And if your use case is that the complexity of GraphQL can really be overwhelming.

Measuring API Success

[00:11:05] Sagar Batchu: Totally. Yeah. There’s like an enhanced simplicity in just exposing your API one resource per, you know, business object or business use case that you want your customers to take advantage of. Yeah I totally resonate with that. You know, just on that API point again, what were some of the metrics you all looked at when you were running these APIs at scale, at Shopify, or even now at OpsLevel?

[00:11:27] What metrics to you, do you think that an API team should measure?

[00:11:32] Ken Rose: Yeah. I mean, I probably would be more apt to talk about what we do at OpsLevel. ‘cause that’s just a lot more recent. I don’t know what Shopify looks at, there’s two classes, right? There’s kinda just the operational metrics, like things like the RED metrics, right?

[00:11:44] Request errors, duration just understand like how is your API performing? For us it’s looking at specific, you know, queries that are coming in and just understand like which ones are, you know, potentially non-performing or what are some areas we have to fix. We have things like, you know, N plus one detection to make sure that like we don’t, you know, explode.

[00:12:00] The other kind of like qualitative one has been feature requests that we get around missing capability in our API. Like I did just finish saying earlier, you know, part of our platform strategy is this sort of Pareto principle. Like we wanna make sure that for problems that are, you know, in the box, like something that 80% of our customer base wants, that’s something we should solve as a first class thing.

[00:12:20] We obviously will have API for it. But we always wanna make sure those APIs exist. ‘cause if we don’t solve for something, you know, there’s an API to do it. That said, sometimes there are things that we will intentionally not have an API for. There’s sometimes that’s due to performance or sometimes it’s, that’s due to the UX is really meant to be more human centered.

[00:12:35] It’s not really meant for like the machine to wanna ingest data or push data to do something with, but we do look at if we’re getting requests for that, how does that translate to the fidelity then of what we’re wanting to do with the API? It becomes sort of that backstop to justify yes, this decision we made to not put this in API maybe that was wrong.

[00:12:50] Or maybe our assumptions about like why we did that were actually correct. Because now we’re getting some input that like, oh, here’s some new use case where they do wanna use API for this particular thing.

[00:12:57] Sagar Batchu: I love that framing because I see a lot of times with companies, they kind of expose a very large API, but then most of the traffic will be on like two endpoints, so one endpoint, and so they don’t really have an understanding of which parts of the API actually getting used. And so they’ve actually exposed a huge part of the API and created a massive dependency for their customers.

[00:13:18] And they move a lot slower as result because they have to update, you know, a hundred endpoints instead of just two, which is like where all the value is. So I love the kind of framing on metrics that you guys have. If, you know, I’m curious if someone is launching an API today and you had to give them like one or two top things to look for.

[00:13:36] What would you advise them on? This is something I see. It’s so funny, I see this on LinkedIn all the time where there’s like API product managers talking about this. They’ll be like the top two or three metrics that an API team should look at. I don’t know if you have a point of view on that.

Key Advice for API Launches

[00:13:48] Ken Rose: Oh man. I mean, I think it depends a lot on the business and like the scale that API is gonna be hitting. Like I could talk about things in general. Like again, think about like your RED metrics, think about versioning of your API and how you’re gonna manage deprecations. It’s almost I like the term API product manager.

[00:14:02] It’s put on your product management hat. What is the value of this API’s providing? Why are your users using it? What are they trying to do? What problem are they trying to solve and how’s your API help ‘em accomplish that? Then just focus on that repeatedly and make it, you know, refine it until it’s as simple as it could possibly be.

[00:14:15] I think that would be like the ultimate lesson. Everything else after that is just details in the pursuit of that goal.

[00:14:20] Sagar Batchu: Yeah, that makes a lot of sense. I think in some ways API platform teams sometimes have to wear that product manager hat for the API. Especially at at larger companies, you have each team ships a part of the API and gets packaged up as like a single library or a single documentation site.

[00:14:37] Unless you have that API platform team owning those metrics and wearing the product manager hat, you often don’t get that thinking. So, yeah, that makes a lot of sense.

API Consistency Across Organizations

[00:14:44] Ken Rose: There is actually one now that you mentioned the API platform team. I do think consistency is an important thing, especially when you are dealing, if you have a large company, you have disparate parts of the company working on different parts of the API. You don’t want to have, is it Conway’s Law that you can kind of, you know, you see the shipping of the, you don’t want Conway’s Law to appear in your API.

[00:15:01] So making sure that API platform team or someone is providing guidance on like here’s how our organization thinks about the shape of APIs. Here’s how you should think about structuring requests. Here’s how you should name parameters. Here’s what response format should look like. Here’s consistent context of return and responses.

[00:15:14] Here’s how to do a pagination. But it just ensuring that consistency because the most frustrating thing as a an API client is okay, I hit this REST, hit this one end point. It works this way. Hit this other point. Why is this structured differently? Like, why does this feel different? Yeah, that can be tremendously difficult.

[00:15:27] So just, you know, having some guardrails or mechanisms to ensure consistency across an API surface area. Also really valuable.

[00:15:34] Sagar Batchu: Yeah, absolutely. I think I’m so glad you brought up, it’s like the number one thing that I see when I look at like an API, that’s old, like a legacy spec, and since we deal with specs I look to the specs and I see I think I can literally draw the org structure by looking at that schema document and being like, “Hey these two are designed by one team, these two are designed by another team. And then this one was tacked on later, right?” Like this such a clear history evolution built into it. No I love that analogy.

AI and the Future of APIs

Switching gears a little bit, I mean, you know, I don’t know if you’ve been following things, but this week I feel agentic AI workflows for API has just really hit its stride. Anthropic’s MCP release, which has been out for a while you know, is kind this protocol around how an AI and agent can access an API. I really, you know, kind of took off on social this the last two weeks. I’m curious what’s your take on all this? Like, how are you seeing, maybe starting with your customers and you know, your ecosystem? Like, are people asking for different tools? How’s the landscape changing for your own API.

[00:16:34] Ken Rose: Yeah, there hasn’t been this like sea change effect from our customer base. “Oh my god, OpsLevel, please start adding more agentic action,” you know? We’re definitely seeing like drips and drops. I think there’s more from like the platform providers like Anthropic.

[00:16:46] Funnily enough my big thing this last week on social was like, there was something called Agents.js that I saw, which seems to solve a similar challenge that describing an API to an LLM is still squishy. And if you just describe the endpoints, the LLMs will hallucinate or make up particular workflows which might not actually make semantic sense.

[00:17:05] You know, something I think a lot about is just I think you’re gonna see more people being able to access APIs that previously couldn’t. Like even now when we talk about APIs, it’s okay, we’re all a bunch of geeky nerds writing curl commands or using postman or whatever it is, right?

[00:17:21] Or writing SDK code. But it’s no, no, no. APIs are gonna become more accessible to, I think a larger audience because it’s wrapped behind a natural language query with an LLM sitting in between it. But then that becomes really hard because LLMs still kind of suck at figuring out the shape of APIs and what APIs to call and what chain to do with them.

[00:17:39] So some of the things that you were describing with Anthropic or Agents.js, being able to have some layer that can better describe to an LLM, “Hey, here’s what you can do that will be deterministic.” Because that’s kind of the heart of what a sequence of API should support.

[00:17:52] It’s gonna be some necessary step for, you know, this technology to sort of be useful, right? I can tell you from personal experience in trying to use LLMs to help my own code writing go faster. I’ll get endpoints that don’t exist. I’ll get chains of API calls that just don’t make sense or don’t compose well together.

[00:18:10] So there’s still I think, a gap in terms of that kind of semantic understanding that LLMs have of the shape of APIs. And you know, I said, and at scale, that becomes really hard when you get really, really large APIs. And I think, by the way, the net effect of all this is that, so more people using APIs kind of larger market that’s gonna mean that there’s gonna be, I think a higher prevalence on like performance and efficiency of APIs.

I remember this is, maybe I’m dating myself with this like 20 years ago, like website operators, like screaming at Google. Like, “why aren’t you satisfying, you know, robots.txt, like you’re scraping my page too often.” I think you might see a similar thing with API providers, you know, dealing with LLMs that are now hitting their APIs way more just because now there’s just, you know, larger numbers of users trying to get access to data.

[00:18:52] Sagar Batchu: Yeah. Yeah, that makes sense. I think, I’ve kind of observed a similar or I’ve had a similar experience. This week I’ve been spinning up MCP servers for any API can get my hand on. And when it’s these like simple, clean APIs it does really well, I can interact with them to Claude or to custom AI and everything kind of just works.

[00:19:10] It feels magical. I was doing it with an email provider and they had just a, they have very simple API just send an email.

[00:19:16] Ken Rose: Send email. Yeah.

[00:19:17] Sagar Batchu: Send email and it works. And it is really cool, you know, type out that email and these clients and then see it just go off. But then I had a point at our own admin API, which, you know, we’re an API company, we like to think we design APIs reasonably well.

[00:19:31] And it did okay, but it wasn’t like at the point where I could be like, “Hey, I can kill our Retool dashboard ‘cause this is just way better.” It’s not quite there. But and I agree with you. I think that is like a semantic, understanding that’s missing. We have this theory that’s because that’s the biggest gap, like the real winner in the space is going to figure out how to do evals and like the optimization problem around schemas. I think what makes like a good API schema for your docs doesn’t necessarily make a good API schema for an agent on LLM to go use.

[00:20:00] So, there’s definitely a gap there. So, yeah I very much see it the same way as you. I think what’s interesting to me as well is that in that space there’s not just APIs for so I should say agents and tools for the common APIs that we all know about, but there’s also ones for the internal APIs.

Unlocking Internal API Potential

[00:20:17] The kind of hidden APIs that have always been hard to work with for companies, right? You go into a big company and I feel like this is part of the value prop of OpsLevel gives me amazing visibility into my own landscape of API as a big company. And I’m actually really excited for, I think agent tools to really like, have an impact there, because that’s really where the like discoverability visibility problem exists.

[00:20:40] I think, you know, working with a public email, API or your notion or GitHub is cool, but it’s, those APIs already be like really well documented and there’s a lot of tools around them. So this is, it’s like adding an extra 5% productivity. But in the internal landscape, I feel like we’ll be able to double productivity if we figure out how to make these things work.

[00:20:57] Ken Rose: Oh a hundred percent. Again, you know, I started the show talking about a bit about like OpsLevel, like we solved this problem with complexity. One of the aspects of that complexity is that you have this fragmented ecosystem of APIs kind of across the board. Some of our customers take very intentional API first strategies where there is an effort around ensuring that those internal APIs aren’t just some artifact of like operations or some artifact of “well, we’re creating a bunch of microservices and they all need to talk to each other, so we’re gonna have APIs.”

It’s “no, no, no. We’re actually gonna ensure that there’s proper documentation. We’re gonna ensure that documentation is consolidated. We’re gonna make that documentation easy to discover for developers.” So that part of that complexity of if I’m a developer writing some feature, needing to know which API, who do I talk to get more information about it? What are the request response parameters? You know, what if something doesn’t go right, how do I debug that? But that’s all sort of in a single place.

[00:21:43] And then, yeah, to your point. Now if you can like superpower that discovery with LLMs to have them generate a lot of that boilerplate for you, but correctly with this massive surface area of internal APIs that exist. I don’t even know if it’s a doubling, I think it’s five or 10x improvement in terms of what organizations can actually achieve.

[00:22:00] Sagar Batchu: Yeah, absolutely. I think the framing on helping you with boilerplate makes sense to me because I think these machines do extraordinary well when you frame the work they have to do around a tight schema of some kind. Whether it’s like an open API schema or it’s a service definition, or it’s even just like a YAML configuration for another product that you’re interfacing with.

[00:22:21] If you have them work around that, then there’s like a sense of determinism and correctness that is easily measured and optimized for. So, yeah I totally agree. If anything, it’s the acceleration of being able to deal with a fragmented landscape. You know, if I have to go look up like five different YAML configs for five different products, but the LLM can do that a lot faster for me, then I’ve you know, just massively increased my productivity.

[00:22:44] Ken Rose: Just look it up, even know where to look or that it even exists out there, right? I can give you another kind of example here. Some of our customers have grown through consolidation, so think about a payments service. They don’t have one. They might have four. So if I’m a developer and okay I’ve been told like, “ah, we’re gonna do something, we’re gonna hit the payment service.” Well, which one which is the anointed one? Which one should I use for this particular use case? Again, having all that context centralized and having it be accessible to LLMs. It just streamlines that entire activity and just makes it go that much faster.

[00:23:13] Sagar Batchu: Oh, I can imagine that’s really useful. Even in legacy API situations like big companies have APIs that do the same thing, but they’re like migrating from V1 to V2 and Yeah. That as a developer you probably have no idea where to look to get this information.

[00:23:27] Ken Rose: Exactly.

[00:23:28] Sagar Batchu: Yeah. And just bringing that into the IDE and into kind of an inline experience, this is so powerful.

What Makes Great Developer Experience?

[00:23:34] Just to wrap things up, I wanted to ask you too you know, you’ve been working with APIs for a while and now at OpsLevel, you’re so close to kind of key developer experience problems. What does great developer experience mean or look to you? Are there like products that you look up to or experiences that have shaped for you what a great DevX is?

[00:23:54] Ken Rose: Yeah. Okay, so I think of DevX similar to like UX, right? It’s like reducing roadblocks to accomplishing a goal. That’s how I think about good user experience. And I think the same is true for developer experience, right? Ultimately it’s an exercise in reducing friction. So just let devs get to the happy place of shipping business value and all, you know, across those three phases of kinda like whether they’re building their software and that kind of inner loop of writing code, whether it’s debugging software, whether it’s shipping software and having clear guidance also on what to do on what is the way to write good code? Whether ‘cause different organizations will bias us for different things, but just making sure that’s very clear I think is an important part of developer experience.

Frequently Asked Questions

What is OpsLevel?

OpsLevel is an internal developer portal that helps engineering teams manage the complexity of large software environments.