Rate Limiting with NGINX and Kratix

Platform engineers work hard to manage traffic and protect applications that are critical to business operations. When these applications receive too much traffic, they can slow down or crash completely, causing poor user experience and service outages. 

Most platform engineers turn to NGINX Ingress Controllers to properly manage this traffic amidst other alternatives. However, configuring and upgrading these controllers at scale requires a lot of manual configurations because as the number of environments increases, the configuration files become large, leading to errors across the cluster.

To solve this, these platform engineers can leverage Kratix, an open-source framework for building internal developer platforms, to automate the deployment, management, and maintenance of NGINX Ingress Controllers using abstractions defined as "Promises" to ensure consistent configurations.

The Kratix framework uses the GitOps workflow and Kubernetes native capabilities to automate the deployment of NGINX Ingress Controllers. Rate limiting is one of NGINX's most valuable yet underrated features. It allows you to limit the number of requests or connections users can make to your server, ensuring these servers aren't overloaded with excessive requests.

In this article, we'll explore rate limiting, different rate limiting strategies, and how to deploy NGINX Ingress Controllers using Kratix. We’d also implement rate limiting features to improve application security within Kubernetes environments.

What is rate limiting in NGINX?

NGINX rate limits use the "leaky bucket algorithm" common in telecommunications to handle traffic bursts while maintaining a steady request flow when users exceed the bandwidth. When the processing queue is filled with requests, this algorithm ensures that these requests are processed according to the first-in-first-out (FIFO) schedule.

Rate limiting prevents brute-force attacks from bots sending thousands of requests per second to your Kubernetes environment. Brute-force attacks involve trying different username and password combinations, increasing the risk of a security breach. NGINX tracks each request or connection and only allows a set number within a given time. It blocks further attempts and returns an HTTP 429 status code (Too Many Requests).

Benefits of rate limiting

As we mentioned earlier, rate limiting is essential for managing traffic flow and protecting systems from excessive requests. This section explores the key benefits of implementing rate limiting to your applications at scale.

• Saving costs

Rate-limiting tools help platform engineers reduce costs and avoid unexpected charges. They control the amount of traffic that comes in and prevent servers from using too much CPU and memory.

• Blocking brute-force attacks

As mentioned earlier, hackers might try to guess your system's passwords and usernames to break in. This is called a brute-force attack. NGINX rate limiters make logins safer by stopping too many failed login attempts.

• Preventing DoS attacks

When attackers flood your servers with too many requests, they can make your app stop working. This is called a denial-of-service attack. NGINX can block these attacks with rate-limiting tools to keep your system running.

Rate limiting strategies with NGINX

• Rate limiting by requests

This method caps the number of times a client can ask a service for something in a set time. For example, you might only let one IP address make 100 requests each minute.

To set this up, use nginx.ingress.kubernetes.io/limit-rps or nginx.ingress.kubernetes.io/limit-rpm annotation. This tells the system how many requests each IP address can make per second or minute.

• Rate limiting by connection

This method caps the number of links a client can open to a server. It helps when clients open too many connections at once to access resources, which can use up server power and block others.

For example, you might limit one IP address to having 10 connections open at a time. To do this, use the nginx.ingress.kubernetes.io/limit-connections annotation to set the maximum number of connections allowed per IP address.

Benefits of using Kratix to manage NGINX installations

With Kratix, managing NGINX becomes easier and more efficient. It helps organisations reduce manual work, keep configurations consistent, and upgrade ingress as a fleet. Below are the key benefits of using Kratix to manage NGINX installations.

• Kratix lets teams self-serve and deploy NGINX

With Kratix, developers can self-serve and deploy services like NGINX to any of their dev environments. This way, these development teams can test how things work together correctly and in the same way, each time, which makes building software better.

Kratix supports fleet management, allowing you to deploy resources to 100s of clusters from the central platform. It also allows platform teams to deliver pre-configured, consistent, and reliable Kubernetes resources. It does this through the Kratix Promise, a YAML document defining what will be installed, configured, and deployed on the Kubernetes cluster. 

In the example NGINX Ingress Promise that can be found in the Kratix marketplace,   there is no need to request individual resources once the Promise is installed. Instead, this Promise is configured to deploy a single NGINX instance per cluster node. This may not be a typical production setup, but it is a good way to get started with exploring Kratix and NGINX.

• Kratix puts all compliance rules in one place. 

Kratix enables platform engineers to provide continuous compliance in the development environment, ensuring the software development processes fully adhere to both industry standards and regulations and internal compliance requirements.

With Kratix, a platform engineering team can enforce policy (such as mandatory TLS or OIDC login). Kratix supports the integration of PaC (Policy as Code) tools, such as Open Policy Agent and Kyverno, that check the compliance of the NGINX ingress Controllers and ensure that proper policies are implemented. 

• Easy upgrading of NGINX versions across the fleet

Kratix makes upgrading NGINX versions across the fleet of environments easier and more consistent. With Kratix, platform engineers can define version updates within a Promise and roll this out across the fleet. All NGINX clusters receive the latest updates without manual intervention.

Implementing rate limiting with NGINX and Kratix

In this section, we will set up everything needed to deploy an NGINX Ingress Controller with Kratix. We will set up the Kratix NGINX Ingress Controller Promise and deploy whoami managed by Traefik, as a sample application. We’d also implement rate-limiting features on the NGINX Ingress Controller to control the traffic coming into the application.

Prerequisites

To follow along with this demo, ensure you set up the following prerequisites:

• Docker: Required for deploying containerised workloads. 

• KinD (Kubernetes in Docker): This is a lightweight solution for running Kubernetes clusters locally. You can also use other tools, such as Minikube or K3s. 

• Kubectl: The Kubernetes command-line interface used to interact with your cluster. Kubectl is used to manage workloads and deploy resources.

• Hey: This is a useful tool for load testing and testing the rate-limiting features

NOTE: If you prefer a cloud-managed Kubernetes cluster like EKS, AKS or GKE, please refer to the Kratix installation guide for more instructions.

Step 1: Install Kratix

Set up Kratix on a single cluster by following the steps in the official guide.

Step 2: Set up the NGINX Promise

With Kratix installed, we will use the Promise from the Kratix marketplace to deploy an NGINX Ingress Controller to the instance. Clone the directory locally and run:

```kubectl apply -f https://raw.githubusercontent.com/syntasso/kratix-marketplace/main/nginx-ingress/promise.yaml```

You can look at the Promise using your favourite editor and see the following:

As we mentioned earlier, this Promise deploys a single NGINX instance per cluster node, even though Kratix Promises are traditionally used to provide things as-a-service. Its multi-cluster scheduling features mean it can also be used to define foundational resources, such as ingress, service meshes, etc. 

After installing the Kratix Promise, it installs the NGINX Ingress Controller, which is exposed as a NodePort service on port 31338. This is not recommended for production use.

Additionally, you can specify prerequisites that must be present on the clusters before deploying NGINX by adding it to the spec section of the YAML file. 

For instance, the NGINX ingress Controller may require some observability features before it works on the cluster. You can specify this extra configuration under dependencies in the spec section of the Kratix Promise YAML file.

Step 3: Verify the installation

Check if everything is working correctly and that the NGINX Controller is running by using these commands:

kubectl get deployment

kubectl get pods 

You should see the following output:

Kratix handles the entire deployment process. It uses GitOps tools like Flux to ensure your cluster matches what's defined in your Promise. With this GitOps approach, Kratix saves the NGINX setup to a Git repository, which starts the deployment pipeline. 

Flux watches this repository and automatically updates your Kubernetes cluster when changes happen, keeping everything consistent and tracking all infrastructure changes. This setup makes fixing problems easier and lets you roll back changes if needed.

Step 4: Deploying the whoami sample application

After setting up the NGINX Promise, we’ll test the functionality of the NGINX Ingress Controller on a sample application. For this, we’ll set up the whoami application as the sample application on the cluster. 

For demo purposes, write a Kubernetes setup consisting of deployment and service manifests to deploy the whoami sample app and create a path so users can reach the application through NGINX.

Create a Kubernetes deployment for whoami

Create a service for whoami

Apply the deployment and service

kubectl apply -f deployment.yaml

kubectl apply -f service.yaml

Step 5: Expose the application via NGINX Ingress 

Create an Ingress resource

Assuming your NGINX ingress controller is exposed at IP 203.0.113.25 (replace this with your actual IP):

Apply the Ingress resource

kubectl apply -f whoami-ingress.yaml

Test the setup

Run the following command to test if everything is working (note: You must replace 203.0.113.25 with the actual public IP of your cluster or Ingress controller.)

curl http://whoami.203.0.113.25.nip.io

Step 6: Modify the Ingress resource to add rate limiting

Apply the updated Ingress resource 

kubectl apply -f whoami-ingress.yaml

Test the rate limiting feature

Test the rate limiting feature by sending multiple requests using hey with the command below:

hey -n 50 -c 5 http://whoami.203.0.113.25.nip.io

Check NGINX Logs by running the following commands:

kubectl logs -n nginx-ingress -l app.kubernetes.io/name=nginx-ingress -f

You should see the following output:

In the image above, the IP address sends multiple requests to the server. The first five requests return a 200 OK status code, and the sixth request shows a 429 Too Many Requests status code.

This is proof that we've been rate-limited! When requests exceed the rate limit, they will return 429 Too Many Requests.

Best practices for implementing rate limiting with NGINX and Kratix

Setting up rate limiting on NGINX helps keep apps safe in the cluster by controlling the amount of traffic that comes in. Here are good ways to set it up.

• Understand traffic patterns

Before you set limits, know how people use your application. Platform engineers can find busy times by watching and studying traffic. This helps you set limits that work for regular users and not block them.

• Set appropriate rate limits

Set up rate limits in your NGINX Ingress Controller to keep things safe and usable. Limits that are too tight can block real users, but limits that are too loose can let in threats. A good way is to set limits based on things like IP addresses.

• Handle traffic bursts

 

During busy times, like when new products come out, NGINX rate limiting helps manage many visitors. This feature lets some extra requests through.

For example, setting burst=10 lets ten more requests in before limiting new ones. Also, the nodelay option processes these requests right away. You can change your settings to include this.

Here, limit-rps sets the rate limit to 5 requests per second, limit-burst allows an additional 10 requests, and nodelay ensures no delay in the burst request.

Simplifying traffic management with NGINX and Kratix: A secure approach.

This article discusses rate limiting and why it's good, different ways to do rate limiting, and how Kratix can be used to automatically deploy and manage NGINX Ingress Controllers to make rate limiting rules work. We also discussed the best practices to set up rate-limiting rules in a cluster.

As a platform engineer, rate limiting helps ensure your applications are secure from outside threats. However, you must determine whether rate limiting is required by thoroughly examining your traffic patterns and the risk of potential security threats.

Visit the Kratix Marketplace for additional information on Kratix and how it streamlines the creation of internal developer platforms. You can also look at the NGINX Promise and other beneficial promises for your platform configuration.