Beyond the Handshake: Scaling WebSockets with Echo

Ludeo is a cloud gaming platform that enable game studios create short, shareable gaming moments called Ludeos, directly from their games. Studios integrate our SDK once, and from that point on, any player in the game can capture and share their best moments. Think of it like a highlight reel, but interactive: anyone can jump in and play the exact moment a creator shared, from any device, no downloads, no installs, no signups. Just click and play - within seconds. We handle the full streaming infrastructure - studios just ship the SDK, and we take care of the rest.

Real-time isn’t a feature at Ludeo. It’s the foundation everything is built on.

Echo is the system that keeps that foundation from cracking. It holds long-lived WebSocket connections between clients and our backend, letting data flow both ways like it owns the place - game-events, live updates, and anything else that refuses to wait politely for a refresh button.

On paper, it sounds simple: “just keep a connection open.” In practice, once you scale past a few clients and servers, things get complicated fast. Suddenly you’re not just “sending messages” - you’re negotiating connection ownership, message routing, delivery guarantees, and the occasional existential crisis at 3AM.

This post is a look at how we got here, and how we traded “it mostly works” for a system with clear ownership, predictable routing, and a delivery model that doesn’t require prayer as a dependency.

The Illusion of Simplicity

Most web applications are built around a simple interaction model: the client sends a request, the server responds, and the connection closes. It’s a stateless pattern that scales well for traditional APIs, but starts to break down when applications need to react instantly to changing state.

Real-time systems have different requirements. Multiplayer sessions, live leaderboards, game-events, collaborative experiences, and streaming updates all depend on the server being able to push data to clients the moment something changes.

Before WebSockets, applications often relied on polling. Clients repeatedly asked the server for updates at fixed intervals. While simple, polling introduces delivery latency since updates are only discovered on the next polling cycle, and creates significant overhead by repeatedly opening HTTP connections, performing handshakes, transferring headers, and closing requests, even when no new information exists. Individually those operations only take milliseconds, but at scale, across millions of requests, the cost starts to compound quickly.

WebSockets changed this model by introducing a persistent, bidirectional connection between client and server. Instead of repeatedly reconnecting over HTTP, the client upgrades a single connection and keeps it open, allowing both sides to exchange messages continuously.

At small scale, this architecture feels straightforward. A client connects, the server keeps the socket open, and events flow naturally between them.

The complexity appears once the system stops being a single WebSocket server. WebSocket connections are fundamentally stateful. With traditional HTTP, any request can be routed to any backend instance. With WebSockets, each live connection remains attached to the specific server that accepted it.

That distinction changes how systems behave during scaling events, deployments, and failures. In a stateless API, instances can usually be added or removed transparently behind a load balancer. Existing requests complete, new requests shift elsewhere, and the system continues operating with minimal coordination.

Persistent connections make this significantly more complex. Removing a node now means dealing with thousands of active clients connected to it. Scaling events introduce connection rebalancing concerns, graceful shutdown becomes a coordinated drain process rather than simply terminating a process, and failures require clients to reconnect and re-establish state against a different server.

Once connection ownership becomes part of the system itself, coordination starts to matter just as much as transport.

Everyone Gets the Message

Before Echo, the real-time system evolved in a fast-paced startup environment, with architecture decisions constantly optimized for speed of delivery. The initial setup used a managed WebSocket provider to move fast, and over time it was replaced with an in-house WebSocket infrastructure as realtime became core to the product.

At that point, WebSocket servers were already consumers of our Kafka event streams, each maintaining its own local map of connected clients. When an event arrived, a server would first check whether it held the target socket locally. If it did, it would deliver the message directly. If not, it would fall back to publishing the event through Redis pub/sub, effectively broadcasting it across the cluster so that every other WebSocket server could check its own local state and attempt delivery.

So routing wasn’t really centralized, it was a simple two-step flow: try locally, otherwise flood the cluster and let every WS server figure it out.

It worked, but as traffic grew, that approach started to show its limits in both cost and predictability. Every message effectively hit the entire system before finding its target.

Consider a cluster with 10 servers. Each message can be sent to all 10 consumers, but only one actually has the target socket. That gives just a 10% chance of hitting the correct server on the first attempt. When it misses - which happens 90% of the time - the message falls back to the Redis pub/sub system, broadcasting across the entire cluster so each server can check its local state and try delivery. Every failed delivery becomes a cluster-wide event. As you add more servers to handle the load, the probability of hitting the right server on the first attempt doesn’t remain at 10% - it drops toward zero, while the cost of each miss keeps increasing.

The deeper issue wasn’t just inefficiency, it was loss of determinism. As the system scaled, it became harder to understand where an event would end up, how many times it would be processed, or what path it took through the system. Delivery became something the system happened to achieve through cluster-wide distribution, rather than something it was directly designed for. That made debugging harder, scaling more expensive, and changes riskier, because no single component had a full picture of message flow.

These gaps in ownership weren’t isolated to routing alone. They showed up across the system in a few recurring patterns:

1. Fan-out to every pod - Without a registry of socket ownership, messages were broadcast across all WebSocket servers, even though only one actually held the connection
2. Centralized delivery bottleneck - Our WebSocket servers consumed a large number of Kafka topics and handled delivery directly, so every new topic meant modifying and redeploying the servers. As topic counts grew, consumer group rebalances became increasingly slow and disruptive
3. No consistent connection lifecycle - Connection handling wasn’t standardized (ping/pong, reconnection, jitter, graceful shutdown), leading to inconsistent recovery behavior across servers and clients
4. Business logic inside WebSocket servers - Routing, deserialization, and subscriptions lived in the transport layer, increasing coupling and limiting scalability, driving throughput down and latency up
5. No shared model - Different WebSocket implementations evolved independently with inconsistent protocols and message formats
6. Limited observability and testing - Issues surfaced, with gaps in logging, metrics, and end-to-end validation

Taken together, these weren’t isolated issues - they all pointed to the same problem: there was no clear model of ownership or routing for realtime behavior, and every part of the system was making its own assumptions.

At that point, it was clear that fixing individual components wouldn’t be enough. We needed to step back and redefine the flow itself: what owns a connection, how events are routed, and how delivery becomes predictable end-to-end.

The Guessing Game Stops Here

Echo is our answer to the chaos. Instead of flooding the cluster and hoping the right server picks up the message, Echo introduces a simple guarantee: Every socket lives on a known server, and every event is delivered directly to it. No broadcasting, no fallbacks, no 10% hit rates. Just a single, deterministic path that does exactly what you tell it to.

Under the hood, Echo is split into two flows: Intake and Broadcast.

Intake handles everything coming from clients - from registering new connections to forwarding client messages into our backend services through Kafka. Broadcast is the reverse path: it takes events produced by our backend services, resolves each one to the exact server that owns the target socket, and delivers it directly.

This separation is what makes routing deterministic: we always know where a connection lives, and we always know how messages flow in both directions.

At a high level, Intake and Broadcast form a continuous loop around Kafka. When a client connects, Echo registers the socket and binds it to a specific server. From that point on, client messages are published into Kafka as events, fully decoupled from downstream services.

Backend services do their thing - firing events into Kafka, completely unaware of which server holds the socket. The Broadcast flow handles the detective work: it checks the registry, finds the right server, and delivers the message straight to its destination.

Don’t worry - we’ll dive into the architecture shortly.

The Backbone: Kafka, and Aerospike

If Echo is the nervous system, Kafka and Aerospike are the spine. We chose them because real-time systems at scale have no room for weak links - connection ownership and message delivery have to be fast, and rock solid, every single time.

Kafka: The Reliable Transport

Every message flowing through Echo - whether it’s a game-state update or a UI notification, passes through Kafka. It’s more than just a pipe; it’s our insurance policy:

• High Availability & Replication: High replication factor, in-sync replicas - if a broker goes down, the event streams don’t even blink. That’s not luck, that’s the kind of redundancy that makes on-call rotations a lot less terrifying
• Decouple everything, lose nothing: Kafka sits at the center of both Intake and Broadcast, cleanly separating transport from business logic. Need to scale? Add more consumers. Need to catch up on missed events? The data is sitting right there, waiting patiently. No drama, no data loss
• Order Guarantees: Partitioning by socket_id means each player gets their own dedicated lane in Kafka. In our SDK play/record flows, messages arrive exactly in the order they were sent - because in real-time gaming, order matters

Aerospike: The Registry

When an event hits the system, we have microseconds to decide where it goes. We use Aerospike as our global socket registry because it offers:

• Predictable Sub-Millisecond Latency: Its hybrid-memory architecture ensures that lookup times stay flat, even if we’ll scale to millions of concurrent connections
• Global Resilience via XDR: Aerospike’s Cross-Datacenter Replication (XDR) is what allows Echo to go global. It synchronizes socket metadata across regions automatically. If a player is playing in eu-central-1, but a backend service in us-east-1needs to send them a message, XDR ensures the US cluster “knows” exactly where that player is sitting without a slow cross-ocean API call
• Secondary Indexes & Lua UDFs: This is how we solve the “zombie connection” problem. If a Kubernetes node disappears, we use secondary indexes on server_addr to find and purge stale registrations in one call

The Architecture

Echo is built as a small set of tightly scoped microservices that implement the two core directions of realtime communication: moving messages from clients into backend services, and delivering events from backend services back to clients. Internally, we refer to these flows as Intake and Broadcast. Each service owns a single responsibility within that pipeline, forming a deterministic path from WebSocket connection to backend processing and back again.

At a high level, the system is built around four core microservices and a shared npm package:

• Echo-Server: The entry point for all WebSocket connections. It handles connection lifecycle management - establishing sockets, maintaining heartbeats, and routing all inbound client messages into the Intake flow after careful validation. It also exposes an internal API used by Echo-Worker to forward events to connected clients
• Echo-Manager: The source of truth for connection ownership. It tracks which server each socket is attached to and provides fast lookups used for routing during Broadcast. It also handles user authentication at the time of WebSocket registration, ensuring only valid connections recorded in the system
• Echo-Router: The ingestion layer that consumes client events from the Gateway (Echo-Server), normalizes them, and publishes them into Kafka for downstream services. It ensures client traffic is fully decoupled from backend processing
• Echo-Worker: This is how backend events are delivered to our clients. It consumes events from Kafka, resolves the target connection via Echo-Manager, and delivers straight to the Echo server that owns it
• @ludeo/echo: The part our developers actually interact with. It’s a lightweight npm package that abstracts away connection setup, serialization, and reconnection logic - exposing a clean, simple interface for publishing and receiving events. The Echo components handle moving the messages; this package ensures schema definitions, constraints, and internal contracts stay consistent without developers having to think about how it all works under the hood

Together, these components turn realtime delivery into a predictable pipeline: clients connect through Echo, messages flow through Kafka, and events are delivered directly to the server that owns the target connection, and back again.

Deep Dive: Intake and Broadcast

Both flows orbit the same idea: a single message, taking a deterministic path. The cleanest way to describe them is to follow one message at a time.

Intake: The client speaks

A client opens a connection using @ludeo/echo, our in-house client library. The library handles the connection lifecycle, query parameters, and exponential backoff on failure. From the client’s perspective, “connecting” is a single Promise; from Echo’s perspective, a lot has already happened.

When the upgrade request reaches Echo-Server, it’s handled manually rather than being delegated to the WebSocket library. Echo-Server first validates the path and query parameters using a strict Zod schema, and only then registers the socket in Echo-Manager - before the WebSocket handshake is completed. That ordering is deliberate. It allows token validation and registration to happen in a single step, ensuring a mapping to the exact pod is written to Aerospike before the client even sees a successful upgrade. If anything fails between registration and handshake completion, the registration is rolled back. No orphaned connections on the way in.

Once connected, every incoming client message is wrapped into a Kafka record with structured headers - socket_id, correlation_id, socket_type, client_type - and published to the intake topic. One detail worth mentioning: for SDK clients, the Kafka partition key is the socket ID, ensuring per-socket ordering. Messages from the same player consistently land in the same partition and preserve order. UI-oriented clients, on the other hand, don’t set a key and are distributed round-robin across partitions. Two ordering models, a single producer path.

That socket_id header isn’t decoration. It’s the breadcrumb that ties an inbound event back to the exact connection it came from. Downstream backend services don’t have to guess “who said this?” - the answer is right there in the Kafka headers. And when those services later want to reply, they have everything they need to address the response back to the same socket: drop socket_id into the broadcast envelope’s data.context, and Echo-Worker takes it from there. Same identity, full round trip.

Echo-Router consumes from the intake topic and branches based on socket_type. This separation is intentional - by moving message deserialization and validation out of Echo-Server and into Echo-Router, the gateway is freed from the cost of opening every message. Echo-Server stays lean, focused purely on connection handling, while Echo-Router does the heavy lifting: decoding, validating, and routing. The result is higher throughput and lower latency at the gateway layer, because the two concerns never compete for the same resources.

SDK binary messages are decoded using FlatBuffers, with a few necessary safeguards like splitting oversized state payloads across multiple Kafka messages to stay within Kafka broker limits. UI text messages are parsed as JSON and validated with Zod. In both cases, Echo-Router ensures socket_id is present, adding it if the client didn't provide it, so routing identity exists in both the Kafka headers and the message body.

From there, normalized events are published to the appropriate downstream topic.

Broadcast: The backend speaks

The reverse path begins with a backend service using @ludeo/echo to publish a JSON envelope to our Kafka broadcast topic. The envelope structure is identical to what a WebSocket client would send - both sides use the same envelope, so clients and backend services share an identical wire format.

export type Envelope = {
  v: string;                                                     // Envelope version - lets us evolve the format without breaking consumers
  type: string;                                                  // Discriminator for the receive handler
  id: string;                                                    // UUID, unique per envelope - useful for dedup/tracing
  ts: number;                                                    // Epoch millis at construction time - powers RTT measurements (e.g. ping/pong) and latency histograms
  data: {
    context: {
      topic?: string;                                            // Intake-only - the Kafka topic Echo-Router should publish to. Irrelevant on the broadcast path
      event_name: string;                                        // Logical event name (e.g. "match.score.updated") - how downstream consumers discriminate
      event_id: string;                                          // UUID for this specific event instance
      correlation_id: string;                                    // Trace id - propagated to Kafka headers and the HTTP X-Ludeo-Request-Id; lets us follow one logical request end-to-end across services
      socket_id: string;                                         // The connection this message is from (intake) or to (broadcast). Stamped by Echo-Server on intake; Set by producer on broadcast to target the socket by ID
      origin: string;                                            // Producer identity - which service emitted this. Mostly relevant on broadcast (e.g. "match-service", "leaderboard-service")
      [key: string]: string | number | boolean | undefined;      // Escape hatch for app-specific routing/trace fields - typed loosely to keep the envelope generic
    };
    payload: any;                                                // Application-defined content
  };
};

Echo-Worker consumes the broadcast topic. For simplicity, we’ll focus on the most direct case: a message targeted at a specific socket via data.context.socket_id - one connection, one delivery.

Echo-Worker queries Echo-Manager to retrieve the registration, performed as a single primary-key lookup in Aerospike - typically completing in sub-millisecond to a couple of milliseconds in-region. The result is a record containing the critical server_addr - the IP and port of the exact pod that owns the connection.

Then comes the part that makes routing deterministic in practice: Echo-Worker sends a direct call to the target pod at http://{server_addr}/websockets/{id}/send. No service mesh, no load balancer, no broadcast layer - just a point-to-point call into the exact Echo-Server that owns the socket. Echo also handles fan-out, where a single event needs to reach multiple clients at once. We’ll save that one for its own post - it deserves the space.

The Server Layer: Lifecycle, Shutdown, and Reality

Echo-Server sits at the entry point of the system, managing thousands of long-lived connections simultaneously. It’s the layer where every detail of the WebSocket protocol matters, every edge case surfaces, and every shortcut eventually turns into a 3AM incident.

The design is driven by four core ideas: owning the lifecycle, dual-layer heartbeats, intentional reconnect cycles, and graceful shutdown as a coordinated process.

We didn’t want a black box

Echo-Server uses the ws library with three deliberate “we own this” toggles:

import { IncomingMessage, Server } from 'http';
import { ..., WebSocketServer } from 'ws';

class WsServer {
  ...

  constructor(server: Server, ...) {
    this.wsServer = new WebSocketServer({
      autoPong: false,                // We handle pong manually to preserve RTT measurement and metadata.modified updates
      noServer: true,                 // We intercept HTTP upgrades ourselves to register sockets in Echo-Manager before the handshake completes
      maxPayload: env.WS_MAX_PAYLOAD, // Hard cap on frame size to prevent memory blowups from malicious or misbehaving clients
      clientTracking: false,          // We maintain our own socket registry, avoiding duplicate state and a second source of truth
      ...
    });
  
    ...

    server.on('upgrade', ...);        // We handle WebSocket upgrades manually

    ...
  }
}

noServer: true means we handle the HTTP upgrade ourselves, which allows us to register the socket in Echo-Manager before the handshake completes. autoPong: false disables silent automatic ping responses, because we rely on those frames to measure RTT and update the modified marker that drives the heartbeat loop. clientTracking: false means we maintain our own socketId-to-WebSocket mapping, which is exactly what Echo-Worker queries during HTTP delivery.

The library handles the protocol; we handle the lifecycle. Each of these flags is a small “we take control” decision - and together, they’re what make the rest of this system possible.

Two ping/pong stories, by design

Echo-Server runs a heartbeat loop that periodically pings every active socket with a timestamp payload. When the client responds with a pong, we measure RTT, refresh the modified marker, and continue. This is the native WebSocket control-frame channel - efficient, protocol-level, and invisible to JavaScript in the browser.

That invisibility is the tradeoff: the server can observe liveness, but in browsers the native ping/pong protocol is handled at the WebSocket layer, below anything the client can access. The client can’t see those frames, measure RTT, or reliably detect when a connection has gone stale. So we introduce a second ping/pong flow over the data channel: a JSON {"type":"ping"...} message that the @ludeo/echoclient can explicitly handle. Same idea, different layer - this gives us visible, application-level liveness and reconnect logic on the client side.

On top of that, the same heartbeat loop enforces a maximum connection lifetime. Once a socket crosses a configured threshold, Echo-Server closes it with code 1001 and lets the client reconnect. This introduces intentional connection resets, preventing long-lived sockets from silently drifting out of sync over time. It also gives us a natural way to redistribute WebSocket load across servers, improving balance as the system scales.

Shutdown is a choreography, not a kill -9

When Kubernetes sends a SIGTERM. The naive response is: the process exits, sockets drop, and clients reconnect. In practice, clients don’t notice immediately - there’s a short delay before they detect the disconnect, and then they all reconnect at once. That spike lands directly on the other pods, which takes the hit. So Echo-Server treats shutdown as its own protocol.

on SIGTERM:

  wait SERVER_SHUTDOWN_DELAY              // Let k8s LB stop routing to traffic us
  server.close()                          // Stop accepting new HTTP requests
  ws.close()                              // Refuse new WS upgrades

  sockets = snapshot(active_sockets)      // Freeze current connection state
  batch_size = floor(
    len(sockets) *
    SHUTDOWN_BATCH_SIZE_PERCENTAGE
  )                                       // Scale batch size with load

  for batch in chunks(sockets, batch_size):
    for s in batch:
      s.close(1001, "Going Away")         // Signal client to reconnect

     wait CLOSE_BATCH_DELAY_MS            // Avoid thundering-herd reconnect

  deadline = now() + FORCE_TERMINATE_MS   // Max time allowed for graceful shutdown

  while active_sockets and now() < deadline:
    wait 1s                               // Give in-flight closes time to settle

  for s in active_sockets:                // Anyone still hanging on...
    s.terminate()                         // ...Gets forcefully disconnected

  cleanup(server_addr)                    // Evict leftover registry state in Echo-Manager
  exit(0)

The 1001 close code is the same “please reconnect” signal we use when a connection reaches its lifetime limit, so client behavior stays consistent whether we’re scaling down or rotating long-lived sockets. (What the client does with that signal is the next beat.)

And when the choreography fails - because reality, again - we fall back to a safety net. Pods don’t always get the chance to shut down gracefully: OOMs, kernel panics, or hard kills happen. When they do, the shutdown flow never runs, and Echo-Manager (backed by Aerospike) can be left with stale registrations pointing at dead servers.

To handle this, a periodic cleanup job calls a cleanup endpoint that triggers a reconciliation flow: Echo-Manager reads all stored server mappings from Aerospike, compares them with the live set of Echo-Server pods from Kubernetes EndpointSlices, and removes anything that no longer exists.

One endpoint, two flows - graceful shutdown for the happy path, and periodic reconciliation for everything else. Either way, stale state doesn’t linger.

The client side: Built for the real world

Server-side resilience is only half the story. Connections drop, networks stutters, laptops sleep, and browser tabs get aggressively throttled. The @ludeo/echo package is built with that reality in mind.

Auto-retry is enabled by default. Any unexpected close triggers a reconnect loop with exponential backoff plus ±20% jitter, so clients dropped by the same event don’t all reconnect at once and overload the system. There’s no max-attempt limit - the loop continues until it reconnects, the client explicitly disconnects, or it receives a non-retriable “close” code like 1000 (clean close) or 1003 (unsupported data).

The result is the difference between a temporary network blip being invisible, and the same blip turning into a support ticket.

We also handle browser tab throttling by design. Background tabs can delay or pause activity in ways that would normally break naive connection assumptions. The package is built so that players on Ludeo can switch tabs freely without random disconnections or false “disconnected” states.

The combination - unsynchronized retries, tab-throttling resilience, and explicit server-driven reconnect signals - means Echo stays reliable in exactly the situations where most WebSocket systems quietly fall short.

It was worth it!

The old system came with its share of bottlenecks, reconnect storms, fan-out floods, and operational headaches. All of those issues are now gone.

• RESTful by design - Every internal API across Echo is RESTful, part of a broader shift toward becoming API-first. Clean contracts, predictable interfaces, no tribal knowledge required to integrate
• Routing went from a cluster-wide pub/sub flood to deterministic per-socket delivery - a registry that knows exactly which pod owns each connection, no guessing, no broadcasting to everyone and hoping for the best
• State stopped living in every WebSocket server’s own little map and moved into a single Aerospike-backed registry with fast secondary index lookups and a Lua UDF powering the global cleanup path. One source of truth, everywhere
• Topics used to mean redeploying our WS servers every time a new event entered the picture. Now Echo-Workers simply listen to the broadcast topic - one topic, multiple partitions. New event flows in, a worker picks it up, no redeploy, no drama
• Lifecycle - It was chaos of inconsistent ping/pong, reconnect, and shutdown behavior scattered across services. Now it’s one set of conventions: heartbeats and batched shutdown baked into Echo-Server, backoff with jitter and inactivity-driven pings baked into @ludeo/echo
• One package for everything - Backend services and frontend clients both use @ludeo/echo. The Kafka producer and the WS client share the same envelope, so the wire format is identical in both directions
• Observability that actually tells a story - Every service ships Prometheus metrics: RTT histograms, per-message latency, connection duration, message sizes, even a shutdown-duration histogram on Echo-Server. End-to-end timings instead of “something is slow, somewhere”
• Alerts that fire on the right things - Kafka consumer lag, processing-time SLOs, error-rate budgets, non-responsive terminations, shutdown overruns, error-logs. If something is drifting, we know before our customers do
• Multi-AZ for HA - Every Echo component runs across multiple availability zones. A zone going down is a non-event: sockets reconnect, Aerospike keeps serving, Kafka consumers rebalance. The system is designed to survive

Echo replaced a fragile WebSocket layer with something designed to scale from the ground up - and every component scales on its own terms. No single bottleneck, no forced coupling. Routing is deterministic, state lives in one place, and the connection lifecycle is consistent across every service. New event types flow in without a redeploy, and a single shared package means backends and frontends speak the same language on the wire.

The system is observable end-to-end, not just “something is slow, somewhere” - and alerts fire on things that actually matter before customers notice. Multi-AZ deployment means a zone going down is an inconvenience for the infrastructure, not an incident for the team.

Is it perfect? Not yet. The foundation is strong, the gaps are known, and the next improvements have a clean surface to build on.

Foundation, Not a Finish Line

The biggest win isn’t a benchmark. It’s that routing is no longer something that happens to us - it’s something we control. We can take any message, trace its path end-to-end, and understand exactly why it ended up where it did. That level of visibility turns debugging from archaeology into reasoning, scaling from guesswork into engineering, and 3AM incidents into short, predictable fixes.

None of the ideas in Echo are novel. Exponential backoff, partition keys, batched shutdown, registry-based routing, and more are all well-worn technics. The work was in composing them into a system with a single, concrete constraint: a delivery model that doesn’t require prayer as a dependency.

The real shift is deterministic ownership. “Who owns this socket?” used to be a philosophical question - now it’s a primary key lookup. That change is what makes everything else possible. As for what’s next - a few teasers, no commitments. And honestly? There’s so much more we can do.

Fan-out is already part of Echo, but the full story - one event, many sockets, delivered correctly and efficiently at scale, deserves its own post. Multi-region socket affinity, so each socket lands on a server closest to the backend services most likely to talk to it, keeping delivery local and skipping the expensive cross-region round trip. Connection-aware load balancing, so new sockets always go to the least-loaded Echo-Server, no more blind round-robin. Dedicated dashboards that make the system accessible to everyone, not just engineers with Postman - pull up a socket, trace a delivery, answer your own question. Observability shouldn’t require a PhD in RESTful APIs.

And finally, none of this would exist without the people who built it. A huge thank you to Omer Sher, Eden Feller, Niv Raviv, Shai Moria, and Claude - for the hard work and late nights. This one’s yours too.

If you have any questions, feel free to reach out or drop them in the comments. We’re also hiring backend engineers - and this is just the tip of the iceberg. If building systems like this sounds interesting to you, we’d be happy to talk.