Creating a Multi-Tenant Prometheus Service with Kratix

Let’s say you get Prometheus up and running for one team in your organisation. Then, all of a sudden, multiple requests start coming in from every product, project, and business unit, and they all want their own isolated monitoring setup. Suddenly, what began as a simple and effective way to collect and view metrics quickly turns into a complex mix of brittle scripts, manual processes, and growing pressure on your team.

The reality is that being able to rely on your monitoring stack is no longer optional. Metrics are essential for understanding how modern applications behave, and tools like Prometheus have become critical in cloud-native environments. However, as your organisation expands, so does the challenge. Providing each team with secure, compliant, and self-service monitoring, without overloading your platform engineers, becomes a serious operational hurdle.

That is where Kratix, the open-source platform framework, comes in.

Instead of patching together custom operators or endless YAML files, Kratix lets you expose Prometheus as a reusable and GitOps-friendly service. It enables true developer self-service, reduces risk, and frees your platform team from repetitive manual work.

In this article, you’ll see precisely how to transform Prometheus from a growing operational challenge into a seamless internal platform feature, ready for your organisation’s next stage of growth.

Why this matters

As mentioned earlier, scaling Prometheus for multiple teams is no small feat. Each team typically needs a dedicated instance to monitor their applications. This ensures secure metrics access without risking data leaks or resource conflicts.

Managing this manually often leads to challenges like:

• Configuration overload: Repetitive YAML configuration files for Prometheus instances, monitoring resources, and access control rules.

• Data leak risks: Misconfigured selectors exposing sensitive metrics across tenants.

• RBAC complexity: Balancing permissions for workloads without over-privileging.

• Scalability bottlenecks: Managing multiple instances across clusters eats up engineering time.

Kratix addresses these pain points by codifying Prometheus as a Promise—a reusable API that delivers isolated, compliant observability with minimal effort. This is a game-changer for platform engineers building internal developer platforms (IDPs) that scale without compromising governance.

Understanding Kratix Promises

Kratix Promises are the core of Syntasso’s framework for creating composable, self-service platforms on Kubernetes. A Promise defines what developers can request (e.g., a Prometheus instance) and how it’s delivered via declarative, GitOps-native workflows.

Think of it as a template combining:

• API: A Custom Resource Definition (CRD) specifying the request schema (e.g., app name, metrics port).

• Dependencies: Prerequisites like the Prometheus Operator, installed on target clusters.

• Workflows: Pipelines that automate lifecycle events, such as provisioning or updating resources.

Promises enable platform teams to expose complex services as simple APIs, ensuring consistency, security, and developer autonomy. In this demo, we’ll use two Promises: one to deploy the Prometheus Operator and another to provision tenant-specific Prometheus instances.

Prerequisites

To set up a multi-tenant Prometheus service with Kratix, you need the following:

• k3d / managed cluster / minikube: A lightweight Kubernetes cluster for testing or a managed cluster for production.

• Kubectl: The command-line tool for interacting with Kubernetes clusters.

• Kratix installed: The open-source platform framework to define and deliver Prometheus as a service. You can follow these steps to install Kratix in your cluster. 

Demo: Defining custom Promises for multi-tenant Prometheus

Architectural overview

To deliver multi-tenant Prometheus monitoring, as stated earlier, this demo will be using two custom Kratix Promises: promoperator and AppObservability.

The promoperator Promise deploys the Prometheus Operator cluster-wide, enabling CRD-based management of monitoring resources.

The AppObservability Promise enables teams to request isolated Prometheus instances and monitoring configurations, like PodMonitors, through a self-service API.

With both Promises defined, the architecture now abstracts the complexity of deploying Prometheus and PodMonitor into reusable workflows.

Step 1: Setting up the Prometheus Operator Promise

In this step, we’ll deploy the Prometheus Operator Promise to enable CRD-based management of Prometheus instances in Kratix.

To do this, apply the following YAML manifest (prom-operator.yaml):

Explanation:

• API: The api section defines a PromOperator CRD, which allows platform teams to request Prometheus Operator installations in a standardised way. This CRD acts as the interface for teams to interact with the Promise.

• Workflows: The workflows section specifies a pipeline (promoperator-install) that automates the installation of the Prometheus Operator. It fetches the latest operator bundle from the Prometheus Operator GitHub repository and applies it to the cluster.

• RBAC: The rbac section is critical in Kratix, as it grants the pipeline the necessary permissions to manage Kubernetes resources securely. Here, the pipeline is authorised to create and manage CRDs (for the Prometheus Operator) and monitoring resources like prometheuses and podmonitors. Kratix’s multi-tenant model requires explicit RBAC definitions to ensure pipelines operate only within their intended scope, preventing unauthorised access across tenants.

To see the full manifest, navigate to the prom-operator.yaml file in the GitHub Gist.

Apply the Promise using the command below:

After which, you can see the “Events” taking place as displayed in the image below.

Viewing Kratix events with Prometheus

Step 2: Creating the AppObservability Promise

In this step, we’ll create the AppObservability Promise to enable teams to request isolated Prometheus instances with custom monitoring configurations, such as PodMonitors, building on the Prometheus Operator.

To do this, apply the following YAML manifest (observe.yaml):

Explanation:

• Dependency: The AppObservability Promise depends on the promoperator Promise to leverage the Prometheus Operator for managing monitoring resources.

• API: The AppObservability CRD lets teams specify application details like appName and namespace for customised monitoring.

• Workflows: The pipeline automates namespace creation and deploys a Prometheus instance and PodMonitor per team, enabling self-service monitoring.

• RBAC: Permissions allow the pipeline to manage namespaces and monitoring resources securely, which is critical for multi-tenant isolation.

To see the full manifest, navigate to the observe.yaml file in the GitHub Gist.

Apply the Promise using the command below:

After running the command above, you can see the AppObservability Promise registered with Kratix as in the image below.

Kratix Prometheus Promise

Step 3: Testing with a sample application

To test the setup, we’ll use Frederik Branczyk’s Prometheus example application. You can find the complete code samples in this repository, but we’ve added summarised versions of the manifests below:

The prometheus-example-app.yaml file:

A developer can now request observability for their application by applying the following manifest (app-observability-request.yaml) :

When a developer requests a promise of AppObservability, Kratix will use the name of the application specified in spec.appName to provision the required resources for monitoring the application.

In this demo, those resources will include a PodMonitor and a dedicated Prometheus instance.  

Apply the manifests above using the commands below:

Port-forward the application: To verify the sample application is running, port-forward the prometheus-example-app pod to port 5000 using the following command:

Access http://localhost:5000 in your browser to see the message: "Hello from example application."

Sample Application UI

Port-forward the Prometheus UI: To access the Prometheus UI, port-forward the prometheus-prometheus-example-app-0 pod using the command below:

Access http://localhost:9090 in your browser. Navigate to the "Status" > "Targets" section to verify the PodMonitor status. You should see:

Prometheus Target Group

Visualising metrics: To visualise metrics, go to the "Graph" tab in the Prometheus UI and query http_request_duration_seconds_sum.

This metric, exposed by the sample application, tracks the total duration of HTTP requests. Plotting it as a graph provides insights into application performance. You can see the graph below:

PromQL Query results

Extensibility: Building a comprehensive observability stack

The AppObservability Promise is a foundation for a scalable observability platform. You can extend it to include the following:

• Jaeger for tracing: Jaeger enables distributed tracing to monitor and troubleshoot application performance across services. Integrating it with the AppObservability Promise enhances observability by tracking request flows.

• Grafana integration for dashboarding: Grafana provides powerful visualisation dashboards for metrics and traces, complementing Prometheus. Integrating it with the AppObservability Promise enables teams to analyse observability data intuitively.

• And other observability tools such as OpenTelemetry, Kibana, Datadog, etc.

Transform observability with Kratix’s Promise

This article explored how Kratix transforms Prometheus from a manually configured monitoring tool into a scalable, self-service platform capability on Kubernetes.

We covered how to automate the provisioning of isolated, production-ready Prometheus instances using Kratix Promises. We demonstrated how Kratix enables developer self-service through a streamlined interface for requesting Prometheus resources.

Additionally, we showed how this setup simplifies Day 2 operations, such as scaling and maintaining compliance across tenants.

For more details on simplifying internal developer platforms, visit the Kratix Marketplace to explore the Observability Promise and other valuable promises that can enhance your platform configuration.