Why Kubernetes Exists: From a Python Script to Production Orchestration

Last Updated on August 3, 2026 by Editorial Team Author(s): Ake Originally published on Towards AI. Ai-generated A practical, first-principles guide to the problems Kubernetes solves — and why Docker alone is not enough Part 1 of the Kubernetes for MLOps series TL;DR Kubernetes exists because running one container is easy, but operating many containers across many machines is not. A Python service is simple, but it creates a single point of failure. Virtual machines improve isolation, but they are heavy, slow to start, and prone to environment drift. Docker makes applications portable, reproducible, and lightweight — but mainly solves the single-host problem. Docker Compose coordinates containers on one machine, not across an entire fleet. Kubernetes adds scheduling, self-healing, service discovery, scaling, and zero-downtime deployments across multiple machines. The central idea is simple: you declare the state you want, and Kubernetes continuously works to make the real system match it. What you will understand after this chapter: Why the industry converged on container orchestration, and what problem Kubernetes actually solves — from first principles, not marketing copy. The Starting Point: A Fraud Detection Team You are the sole ML engineer at a fintech startup. The payments team has trained an XGBoost model that detects fraudulent transactions with 94% precision. The model needs to run as a real-time inference service: every card swipe calls your API within 200ms and gets a fraud probability score. If the score exceeds a threshold, the transaction is blocked. The model works. Now the infrastructure becomes your problem. This chapter traces exactly how that problem evolves — from a Python script to a Kubernetes deployment — and at every step explains why the current approach broke down and what each new layer actually solved. Era 1: Start with a Python Service You start the only way an engineer should: the simplest thing that works. # fraud_detector.pyimport numpy as npimport xgboost as xgbfrom fastapi import FastAPIfrom pydantic import BaseModelimport logginglogging.basicConfig(level=logging.INFO)logger = logging.getLogger(__name__)app = FastAPI(title="Fraud Detector", version="1.0.0")# Model loaded once at startup — lives in this process's memorymodel = xgb.XGBClassifier()model.load_model("fraud_model.json")logger.info("Model loaded successfully")...@app.get("/health")def health(): return {"status": "ok"}... You run it: uvicorn fraud_detector:app --host 0.0.0.0 --port 8000 --workers 4 It works. The payments team integrates it. Transactions flow. Life is good for about six weeks. What Breaks Single point of failure. Your process is the only instance. When it crashes — due to a memory leak, an unexpected exception, a malformed input — every downstream payment attempt fails. At 3am on a Saturday. No isolation. The fraud detector shares the OS, filesystem, CPU, and memory with every other process on that machine. A misconfigured apt upgrade can break your Python runtime. A different service leaking memory OOM-kills your process. You have no guarantees. Manual deployments. Retraining the model means SSH-ing to the production server, copying a new fraud_model.json, and restarting uvicorn. Every deployment is a manual SSH session. Mistakes happen. There is no rollback. No horizontal scaling. Transaction volume grows 5x after a marketing campaign. You cannot add capacity without significant manual intervention. The single instance becomes a latency bottleneck. No resource limits. A bug in the feature extraction code causes a tight loop. Your process consumes 100% CPU. Other services on the same host degrade. Era 2: Add Isolation with Virtual Machines The first instinct is correct: isolate services. Virtual machines provide hard boundaries between workloads. The isolation story is real. A crash in VM 1 does not affect VM 2. The hypervisor enforces CPU and memory boundaries. You can snapshot, restore, and clone VMs. You have an audit trail. What virtual machines did not solve Resource waste at scale. A Ubuntu 22.04 minimal install consumes roughly 2GB of RAM just to exist. Your XGBoost model with a FastAPI wrapper needs about 400MB of RAM to serve traffic. The VM tax means you are paying for 2GB of RAM per instance just to run a 400MB application. Across a fleet of 50 fraud-detection VMs, that is 100GB of RAM doing nothing but running OS daemons. Boot time. A VM takes 30–90 seconds to boot. When traffic spikes suddenly — a flash sale, a bot attack, a news event — you cannot add capacity fast enough. By the time a new VM is healthy, the spike has passed. Environment drift. Two VMs provisioned from the same Machine imagesix months apart will differ. Security patches, library updates, and manual configuration changes accumulate. You have experienced “it works on VM 2 but not VM 3” at the worst possible time. Slow iteration. To deploy a new model version, you build a new Machine image(10–15 minutes), launch a new instance (2–3 minutes), wait for health checks (1–2 minutes), shift traffic. A deployment takes 30 minutes minimum. Rolling back is not faster. The dependency conflict problem. The fraud detection service needs XGBoost 2.0. A new anomaly detection service needs XGBoost 1.7 because a legacy dependency pins it. On VMs, both services share the system Python. You either containerize the environments manually (virtualenv, conda) or run each service on its own VM — amplifying the waste problem. Virtual machines solved isolation. They created a new category of problems around density, speed, and reproducibility. Era 3: Package the Service with Docker Docker and Containers: The Essential Concepts Docker did not invent containers. Linux already provided the core technologies, especially namespaces and control groups (cgroups). Docker’s main contribution was making containers easy to build, distribute, and run consistently across different environments. Namespaces: Process Isolation Linux namespaces give a process its own view of system resources. The container can also have its own hostname, filesystem, and network interface. However, it still shares the host’s Linux kernel. cgroups: Resource Limits Namespaces provide isolation, while cgroups control resource usage. With Docker, you can restrict how much CPU and memory a container can consume: docker run \ --memory="512m" \ --cpus="1.0" \ fraud-detector:v1.2.0 This container can use up to: 512 MB of memory One CPU core If it exceeds its memory limit, the kernel can terminate the container’s process without directly […]