What We Learned Building a Prototype AI-Driven Dev Interface for Kratix
- Shane Dowling
- 12 hours ago
- 4 min read
Like many software delivery teams, engineers at Syntasso have been experimenting with ways to integrate LLMs into our internal developer experience. In one of our recent internal hack days, I tried wiring up a Claude Desktop interface to Kratix using FastMCP. The idea was simple: could engineers ask natural-language questions, assemble a platform, and get something deployed without learning YAML or kubectl?
This wasn’t intended to be a polished product build; instead, it was more like a spike to explore what worked and what didn’t. What follows is a very honest reflection on how it went: the surprises, the limitations, and the moments where it all just clicked.
So, are we ready to hand over platform engineering to AI?
The dream here is that someone could chat to a Kratix cluster and actually do stuff to build a platform. For example, deploy Redis, spin up namespaces, maybe even check on the status of a deployment, without needing to touch kubectl.
The tl;dr is that it works. Mostly.
But there were a bunch of lessons along the way that I didn’t expect, so I thought I’d jot some of them down while they’re fresh.
1. The setup is a bit of a faff
The FastMCP project is great once it’s running, but getting there isn’t plug-and-play.
You can’t just point it at an existing FastAPI app or run it under something like Cloud Desktop without using uv and the custom FastMCP.run() entry point. I get why it works this way - it's doing a bunch of code introspection - but it still feels heavier than I expected.
Once I had the right structure in place (and worked around a couple of undocumented gotchas), it was solid. But it definitely wasn’t “just drop it into your repo” levels of easy.
2. You’ve got to be intentional with your endpoints
One of the bigger surprises was how much care you have to take in what you return from each endpoint. The AI isn’t magic. It’s just guessing which tool or resource to invoke based on your prompt and what it knows about the available tools.
If you hand back a list of destinations with slightly misleading keys (like ready being null instead of "True"), it’ll make the wrong call. Likewise, if it sees two endpoints with similar-looking output, it might pick the wrong one based on context you didn’t think mattered.
Moral of the story: LLMs are easily confused by subtly different API responses. Be really explicit in what you expose.
3. The AI is really persistent
This was both cool and frustrating: ChatGPT was determined to answer questions.
If it didn’t have quite the right data, it would fall back to adjacent tools, make assumptions, or just invent the glue logic. Sometimes this was helpful. Other times it meant it did something completely unexpected, like trying to list resources on the platform cluster when it should’ve queried the destination instead.
If you want predictable behaviour, you need to do more than give it endpoints; you need to guide the flow of logic.
That’s a future challenge I haven’t totally solved yet. Maybe we’ll need some sort of middleware for validating assumptions ("Did you mean to delete that Redis instance?") or structuring multistep flows.
4. Debugging is a new type of pain
When something goes wrong, it’s not always clear why. Is it the AI picking the wrong tool? Was the response malformed? Did the underlying Kubernetes query fail? Did the context have the wrong assumptions?
You can log everything, sure - but what I’m missing is a “why did you do that?” button. Like: why did it call fetch_destinations instead of fetch_destination_contexts when I asked "where was Redis deployed?"
That kind of introspection is tricky right now.
5. Kratix Promises appear to be made for this use case!
This whole model works especially well because of how Kratix Promises are structured.
Promises, and the resource requests that flow from them, are already abstractions built by platform engineers who know their org's needs. They’ve already baked in all the assumptions: “Our devs aren’t Kubernetes experts,” or “They just want a working database with an endpoint.”
Because of that, the AI isn’t interacting with raw Kubernetes. It’s talking to a layer designed with business context in mind. The interface is already scoped. It’s safe. It’s shaped to fit the platform team’s contract.
This ultimately means that ChatGPT (or any AI) isn’t likely to go off-piste; it just works within the guardrails we’ve already defined.
6. It’s just another interface - and that’s a good thing
This whole experiment shifted how I think about LLMs.
Instead of seeing them as this all-knowing mega-brain, I’ve started seeing them as just another interface. We’ve got CLIs, mobile apps, dashboards, REST APIs; and now, we’ve got a chat interface. That’s it. It doesn’t need to consume the whole stack or do every task.
Your AI interface should let someone talk to the system in a way that feels natural. It’s a new product surface, and that’s powerful on its own.
7. It’s still pretty cool
Despite the faff and the debugging and the occasional weirdness, there’s something very cool about typing “Give me a Redis” and having the system submit a real Kratix PromiseRequest.
It’s clunky, but it’s a glimpse of how this kind of AI layer could actually help people interface with platforms without needing a terminal, docs, or YAML.
It’s early days, but I think there’s a real shape here. Especially if we can define reliable guardrails and flow patterns.
Wrapping up: AI, just another interface?
This experiment changed how I think about LLMs and AI usage within platform engineering. I no longer think of them as all-knowing agents or dev team replacements, but as another interface to your product, like a CLI, mobile app, or dashboard.
If you already offer abstractions to your internal teams via Promises and resource requests, it turns out that’s a perfect foundation for AI to work well: you define the scope, the AI navigates it.
You can get started with Kratix via our docs. If you’re thinking about AI-driven platforms, I’d love to hear how it’s going, whether you’re a human or a bot.
댓글