Why WebSockets don’t scale easily — and how AWS changes the game

Last Updated on July 6, 2026 by Editorial Team Author(s): Leapfrog Technology Originally published on Towards AI. WebSockets are deceptively simple. Every connected user maintains a persistent connection to the server, and each connection continuously occupies server resources such as memory, CPU cycles, network buffers, and application state. Unlike traditional HTTP requests, WebSocket connections are long-lived and stateful, meaning resource consumption grows almost linearly with the number of users. You can, however, derive a simple mathematical model to introduce the problem. The fundamental problem Every active WebSocket connection consumes server resources. For each connection, a Node.js process maintains: A TCP socket Read/write buffers Event listeners User session metadata Heartbeat/ping state So:Total Memory=N×Mcwhere N = number of concurrent users Mc​ = memory consumed per connection Resource usage assumptions for 1000 concurrent users establishing WebSocket connections The following estimates assume: 1,000 simultaneously connected clients Clients are mostly idle (heartbeat/ping messages only) Small text messages (less than 1 KB) No large binary payloads One Node.js process Linux server TLS termination handled separately (or already accounted for) Figures represent application memory only, not total system memory (These are realistic ballpark figures (actual values vary by implementation). Resource Usage for Raw WebSocket (ws node.js package) implementation The ws library is a lightweight implementation that exposes the WebSocket protocol with minimal abstraction. Typical per-connection memory consumption: Memory: 1000 × 30KB = 30,0000KB≈ 29 MB Resource Usage for Socket.IO package implementation Socket.IO builds on top of WebSockets (via Engine.IO) and adds higher-level functionality such as Engine.IO layer Rooms Acknowledgements Automatic reconnection Packet encoding Additional metadata Typical per-connection memory consumption: Memory: 1000 × 600KB = 60,000KB≈ 58 MB Visual comparison This is approximately linear growth: Memory ∝ Number of Connections But memory isn’t the real problem Memory is only one dimension. Suppose an EC2 instance has: 2 GB RAM 1 vCPU If your application itself consumes: 500 MB for Node.js application code 200 MB for caches 300 MB OS overhead Available for WebSockets: 2GB − 1GB = 1GB Using raw WebSockets: 1000MB ÷ 30KB ≈ 34,000 theoretical connections Using Socket.IO: 1000MB ÷ 60KB ≈ 17,000 theoretical connections In reality, CPU, network bandwidth, and event-loop latency become bottlenecks much earlier (often around 5k — 20k connections per Node process depending on message frequency). The scalability challenge The issue isn’t serving 1,000 users. The issue is serving 100,000 users. If one server holds 10,000 persistent connections: 100,000 ÷ 10,000 = 10 servers Now you must manage: Load balancing Sticky sessions Cross-instance pub/sub Connection failover Redis adapters Auto scaling At that point, WebSockets become a distributed systems problem rather than a networking problem. Scaling WebSockets with Node.js, Nginx, and Redis Pub/Sub To move beyond a single-server bottleneck, WebSocket connections are typically distributed across multiple Node.js instances running in a cluster behind a reverse proxy like Nginx. Figure: High-Level Architecture supporting scalable WebSocket with Redis Backplane. At a high level, the architecture looks like this: Nginx handles incoming WebSocket upgrades and load balances connections across Node.js upstream servers. Each Node.js instance manages only a subset of active connections. A Redis Pub/Sub layer synchronizes messages across all instances. This solves a key limitation: WebSocket connections are stateful, so without coordination, one server cannot notify clients connected to another server. We can model the scaling behavior as: N = n_1 + n_2 + n_3 + … + n_k Where: N = total concurrent connections n_k = connections handled by each Node.js instance If each instance supports ~10,000 connections: 100,000 users ≈ 10 Node instances However, connection distribution alone is not enough. Message propagation requires cross-instance communication: Client A → Node A → Redis Pub/Sub → Node B, Node C → Clients So Redis acts as the “event backbone”, ensuring messages reach all subscribed instances regardless of which server holds the original socket. In practice, the system scales in two dimensions: Vertical (per node): limited by memory per connection Horizontal (cluster): limited by coordination overhead (Redis + network hops) This architecture allows WebSockets to scale from thousands to hundreds of thousands of concurrent connections, but introduces a new tradeoff: distributed complexity replaces single-server simplicity. The operational cost of self-managed WebSocket scaling Once WebSockets are distributed across multiple Node.js instances with a load balancer and a Redis Pub/Sub backbone, the system becomes functionally scalable but operationally heavy. At this stage, scaling is no longer just about handling connections; it requires continuous management of infrastructure behavior in real time. Engineers must now monitor and tune: Connection distribution across nodes (to avoid uneven load) Memory per instance (to prevent socket exhaustion) Redis Pub/Sub throughput and latency Message fan-out patterns across services Network hops introduced by inter-node communication Failure recovery and reconnection storms during outages In essence, the system shifts from a simple connection model to a continuously evolving distributed system where every component becomes a potential bottleneck. The AWS approach: Offloading the complexity Managed AWS WebSocket solutions, such as API Gateway WebSockets, fundamentally change this model by removing the need to manage persistent connection infrastructure directly. Instead of maintaining Node.js servers for connection handling, AWS takes responsibility for: Maintaining persistent WebSocket connections at scale Handling connection lifecycle (establish, reconnect, disconnect) Scaling underlying infrastructure automatically Routing messages to connected clients via managed APIs Integrating with backend services through AWS Lambda, SQS, or EventBridge In this model, engineers no longer manage sockets as infrastructure primitives. Instead, they work with higher-level events: Client Event →AWS API Gateway → Compute Layer (Lambda / Service) → Event Routing Back to Clients What this removes from engineering ownership Moving to AWS-managed WebSockets shifts responsibility away from engineers in several critical areas: No server provisioning or horizontal scaling for WebSocket nodes No load balancer tuning or sticky session management No Redis Pub/Sub coordination layer No manual connection tracking or socket state distribution No handling of connection spikes or reconnection storms No direct memory management per connection Simple 1-to-1 Chat using AWS API Gateway WebSockets This is a minimal architecture for building a real-time 1-to-1 chat system using AWS-managed WebSockets. The goal is to avoid […]