TCP Packet Coalescing: Root Cause and Solutions

Many people treat “packet coalescing” as a TCP bug, but TCP really doesn’t deserve the blame. Understand packet coalescing and how to correctly handle message boundaries at the application layer.

First, packet coalescing is not a TCP problem. The various articles about the “TCP packet coalescing problem” listed in search engines are actually about coalescing caused by poorly designed application-layer protocols that use TCP for data transmission. It is not TCP’s fault.

TCP is a stream protocol and has no concept of packet coalescing.

The so-called packet coalescing problem refers to a situation where multiple messages from the sender are concatenated together when they arrive at the receiver, making them impossible to parse. To solve this problem, you need to design the application-layer protocol properly and agree on message boundaries, so that even if multiple messages are concatenated, the receiver can still locate the boundaries according to the protocol and parse them correctly.

The reason many people regard packet coalescing as caused by TCP is that the “concatenation” behavior occurs at the TCP layer. Data sent and received through TCP is in stream format. When an application-layer protocol uses TCP to transmit data, TCP may split messages sent by the application layer into multiple packets sent sequentially, or combine multiple messages before sending them. As a result, a data segment received by the receiver may consist of multiple messages, which is packet coalescing.

In reality, TCP is a protocol based on byte streams rather than message packets. It guarantees in-order delivery of the byte stream, and parsing the byte stream should be done by the application-layer protocol. Therefore, the packet coalescing problem is actually about “how to design an application-layer protocol”.

graph LR
    subgraph 发送端
        A["消息 A"] -->|TCP| B[字节流]
        C["消息 B"] -->|TCP| B
        D["消息 C"] -->|TCP| B
    end

    subgraph TCP 层
        B -->|可能合并/拆分| E[传输单元]
    end

    subgraph 接收端
        E -->|字节流| F[应用层缓冲区]
        F -->|无边界标识| G["粘在一起：A+B+C"]
    end

    style G fill:#ffcccc

Solutions

The application layer can define message boundaries in the following ways:

Approach	Principle	Suitable scenarios
Fixed length	Every message has a fixed length	Simple, but wastes bandwidth
Length prefix	Send the length first, then the data	General-purpose solution
Delimiter	Separate messages with special characters	Text protocols (such as HTTP)
Hybrid	Length + delimiter	Complex protocols

// 长度前缀方案示例
// 发送端
const send = (socket, message) => {
  const data = JSON.stringify(message);
  const length = Buffer.byteLength(data);
  const lengthBuffer = Buffer.alloc(4);
  lengthBuffer.writeUInt32BE(length);
  socket.write(lengthBuffer);
  socket.write(data);
};

// 接收端
const buffer = Buffer.alloc(0);
socket.on('data', (chunk) => {
  buffer = Buffer.concat([buffer, chunk]);
  while (buffer.length >= 4) {
    const length = buffer.readUInt32BE(0);
    if (buffer.length >= 4 + length) {
      const message = JSON.parse(buffer.slice(4, 4 + length).toString());
      buffer = buffer.slice(4 + length);
      console.log('收到消息:', message);
    } else {
      break;
    }
  }
});

Solutions

Related reading