Rust at NetTV: An 84% Reduction in Infrastructure Costs

NetTV is the biggest IPTV/OTT platform of Nepal and at scale we face a lot of interesting technical challenges. To give a hint of scale, we serve over 500,000 users. It is our sole responsibility to ensure our services provide users with the best experience. This blog presents with one of the issue we faced on production internally and our approach towards solving it.

Running out of memory

Recently, some of our nodes were observed to be the victim of OOM killer. All of these nodes had one thing in common, they were running the same WebSocket service and nothing more. I accumulated conttrack entries across all of these nodes and on multiple occasions, TCP connection entries were 1.5x to 4x more than the number of users online - there should be only one connection per device. Based on our logs, it was evident that here had been no malicious traffic. I assumed our application to be establishing duplicate WebSocket connection for the same functionality. Fortunately, there had been no such cases where this led to any undefined behavior, simply a duplicate connection. 

I inspected the service independently, the functionality of the service was trivial - a nodejs application connected to Redis & Kafka with the purpose of fanning-out messages to end-users. Duplicate connection was definitely concerning but the shock on my face when I saw this service being provisioned a total of 26cores & 52GB RAM. The worst part being, the service still OOM’ed. That's an expensive amount of CPU  and DRAM. At this point we could definitely scale up to solve our problem but let's be honest, NetTV is growing fast and only grows bigger - we need to solve the problem not delay it.

Why not Golang ?

A work stealing runtime such as the Go’s runtime or tokio (rt-multi-thread enabled) moves the ownership of socket across threads at times. WS connections are long lived and ping-pong throughout its lifetime. We benefit by not moving connections across threads because of CPU cache hits. With maximum performance in mind, I decided on a Thread Per Core, share nothing architecture and io_uring over epoll for maximum throughput. I needed to spin up an async runtime backed by io-uring per OS thread and Golang just doesn’t support it.

Rewriting it in Rust

I wrote the initial version on top of the compio runtime, along the rewrite I ran into a bug with WebSocket which at the time remained unsolved. I decided to fallback to using tokio, creating a single threaded (rt-multi-thread disabled) runtime per thread. I also tested different allocators, specifically jemalloc and mimalloc and ended up using jemalloc.

fn create_tcp_listener(config: &Rc<Config>) -> Result<TcpListener, Error> {
    let addr = config
        .ws_bind_addr
        .parse::<SocketAddr>()
        .inspect_err(|e: &std::net::AddrParseError| {
            error!(
                "WebSocket (error: {e}) - failed to parse address: {}",
                config.ws_bind_addr
            )
        })
        .map_err(|_| Error::InvalidConfiguration)?;

    let socket = TcpSocket::new_v4().unwrap();
    socket.set_reuseaddr(true)?;
    socket.set_reuseport(true)?;
    socket.bind(addr)?;

    let listener = socket.listen(1024)?;
    Ok(listener)
}

Each thread binds its own TCP socket to the same address using SO_REUSEPORT (this has no effect on macos but on linux - the kernel load balances traffic across all listeners).

Post Analysis

Did rewriting it in rust solve the problem ? The short answer is yes. It, of course, didn't solve the duplicate connection we were facing but we sure did save ourselves from the OOM killer. Here is a observation we made after the deployment of the rust rewrite onto a single server with 4cores and 16GB memory, handling almost a million active WebSocket connections.

A single server handling all of production traffic is not something you want to do. This was a HA setup and I had multiple standby nodes ready to re-balance traffic.

CPU & Memory usage while handling almost a million active websocket connections.

With this deployment, we were already serving all of production traffic including duplicate connections. Now, let’s compare to see the benefits from the rust rewrite.

84% reduction in CPU cores, it speaks for itself.

For those interested in thread per core, I recommend reading these posts from Iggy.

• https://iggy.apache.org/blogs/2026/02/27/thread-per-core-io_uring/
• https://www.reddit.com/r/rust/comments/1pn6010/compio_instead_of_tokio_what_are_the_implications/