Bar Dror

Posted on Oct 26 Originally published at codebeneath.com

UDP Datagram Size Limits and Fragmentation: When Packets Break

#go #networking #programming

Originally published on Code Beneath.

UDP has no built-in stream reassembly like TCP. When you send a datagram larger than the network can carry in one frame, the kernel fragments it at the IP layer, and any single lost fragment kills the entire message. Understanding UDP maximum datagram size fragmentation example scenarios is critical because they fail silently, often at scale or under bad network conditions, and most developers encounter this only after production incidents. This article walks you through exactly where the limits come from, how fragmentation breaks, and how to measure it with working code.

The Hard Limits

UDP has a theoretical maximum payload of 65,535 bytes (the 16-bit length field in the UDP header). But that is not the practical limit. The real constraint is the Maximum Transmission Unit (MTU), typically 1500 bytes on Ethernet. Your datagram must fit in one IP packet. If it does not, the IP layer fragments it into multiple packets, each with its own header. If any fragment is lost in transit, the entire datagram is lost. There is no retransmission like TCP. Gone.

On a 1500-byte MTU network:

IP header: 20 bytes (minimum, IPv4)
UDP header: 8 bytes
Usable UDP payload per packet: 1500 - 20 - 8 = 1472 bytes

If you send 1500 bytes of UDP payload, the kernel must split it. Packet 1 carries 1472 bytes, packet 2 carries 28 bytes. Both must arrive or the receiver gets nothing. Loss probability scales with fragment count. This is not a theoretical risk: it is why DNS uses TCP for responses larger than 512 bytes, why NTP limits payloads to 152 bytes, and why your game server likely hard-caps message size.

The situation worsens if you traverse tunnels (VPN, MPLS) or go over wireless. Those layers add overhead, shrinking the effective MTU further. On some paths, it can drop to 1280 or even lower.

Measuring MTU with Path MTU Discovery

Path MTU Discovery (PMTUD) is supposed to help. The sender sets the DF (Don't Fragment) bit on outgoing IP packets. If a router finds a packet too large for its next hop, it discards the packet and sends back an ICMP "Fragmentation Needed" message containing the maximum size that path can handle. The sender then reduces its transmission size. This works in theory and often fails in practice: many networks filter ICMP for "security," leaving the sender waiting forever or silently dropping traffic.

You can probe the path MTU yourself using a binary search. Start with a guess (say 1472 bytes), increment until you get nothing back, then narrow the range. Here is a working Java example:

import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;

public class MTUProbe {
  public static void main(String[] args) throws Exception {
    String host = args.length > 0 ? args[0] : "8.8.8.8";
    int port = 53; // DNS
    int timeout = 2000;

    InetAddress addr = InetAddress.getByName(host);
    int low = 100, high = 1472, discovered = 0;

    System.out.println("Probing path MTU to " + host + ":" + port);

    while (low <= high) {
      int mid = (low + high) / 2;
      boolean reached = probeSize(addr, port, mid, timeout);

      System.out.printf("Size %d: %s\n", mid, reached ? "OK" : "FAIL");

      if (reached) {
        discovered = mid;
        low = mid + 1;
      } else {
        high = mid - 1;
      }
    }

    int payloadMTU = discovered - 28; // 20 bytes IP + 8 bytes UDP
    System.out.printf("\nPath MTU: %d bytes, safe UDP payload: %d bytes\n",
        discovered, payloadMTU);
  }

  static boolean probeSize(InetAddress addr, int port, int payloadSize,
      int timeout) throws Exception {
    try (DatagramSocket socket = new DatagramSocket()) {
      socket.setSoTimeout(timeout);
      byte[] data = new byte[payloadSize];
      java.util.Arrays.fill(data, (byte) 0x42);
      DatagramPacket packet = new DatagramPacket(data, data.length, addr,
          port);
      socket.send(packet);

      try {
        DatagramPacket response = new DatagramPacket(new byte[1500], 1500);
        socket.receive(response);
        return true;
      } catch (java.net.SocketTimeoutException e) {
        return false;
      }
    }
  }
}

Compile and run against a DNS server or any UDP responder:

javac MTUProbe.java
java MTUProbe 8.8.8.8

On most commodity networks, this will discover 1472 bytes for IPv4. On IPv6 or behind certain proxies, you may see 1452 or lower. Save that number and use it as your hard cap for UDP payload size.

Fragmentation in Action: A Loss Experiment

Now let us demonstrate the cost of fragmentation. I will build a server that accepts a size parameter, sends back a UDP datagram of that size, and we will measure losses at different payload sizes. We will use two languages to show the pattern is universal.

Java Server:

import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;

public class FragmentationTestServer {
  public static void main(String[] args) throws Exception {
    int port = 9999;
    DatagramSocket socket = new DatagramSocket(new InetSocketAddress(port));
    System.out.println("UDP Fragmentation Test Server listening on port "
        + port);
    System.out.println("Send a message with format: SIZE");
    System.out.println("Example: echo 'SIZE:2000' | nc -u localhost 9999");

    byte[] buffer = new byte[8192];
    DatagramPacket request = new DatagramPacket(buffer, buffer.length);

    while (true) {
      socket.receive(request);
      String msg = new String(request.getData(), 0, request.getLength())
          .trim();

      try {
        int size = Integer.parseInt(msg.replace("SIZE:", ""));
        size = Math.min(size, 65507); // 65535 - 28 for headers

        byte[] response = new byte[size];
        ByteBuffer bb = ByteBuffer.wrap(response);
        bb.putLong(System.nanoTime());
        for (int i = 8; i < response.length; i++) {
          response[i] = (byte) ((i + 1) % 256);
        }

        DatagramPacket reply = new DatagramPacket(response, response.length,
            request.getSocketAddress());
        socket.send(reply);

        System.out.printf(
            "Sent %d bytes to %s (fragments: %d)\n",
            size, request.getAddress().getHostAddress(),
            (size + 1471) / 1472);
      } catch (NumberFormatException e) {
        System.out.println("Invalid format: " + msg);
      }
    }
  }
}

Java Client:

import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;

public class FragmentationTestClient {
  public static void main(String[] args) throws Exception {
    String host = args.length > 0 ? args[0] : "127.0.0.1";
    int port = 9999;
    int trials = 5;

    System.out.println("Testing UDP fragmentation to " + host + ":" + port);
    System.out.println("Size\tFragments\tRecv\tLoss%");

    for (int size = 100; size <= 8000; size += 500) {
      int received = 0;
      for (int trial = 0; trial < trials; trial++) {
        if (sendAndWait(host, port, size)) {
          received++;
        }
      }

      int fragments = (size + 1471) / 1472;
      double loss = 100.0 * (trials - received) / trials;

      System.out.printf("%d\t%d\t\t%d\t%.0f%%\n", size, fragments, received,
          loss);

      try {
        Thread.sleep(100);
      } catch (InterruptedException e) {
      }
    }
  }

  static boolean sendAndWait(String host, int port, int size)
      throws Exception {
    try (DatagramSocket socket = new DatagramSocket()) {
      socket.setSoTimeout(1000);
      InetAddress addr = InetAddress.getByName(host);

      byte[] request = ("SIZE:" + size).getBytes();
      socket.send(new DatagramPacket(request, request.length, addr, port));

      byte[] response = new byte[size];
      try {
        socket.receive(new DatagramPacket(response, response.length));
        return true;
      } catch (java.net.SocketTimeoutException e) {
        return false;
      }
    }
  }
}

In another terminal, start the server:

javac FragmentationTestServer.java FragmentationTestClient.java
java FragmentationTestServer

Then run the client (on the same machine, so no real loss):

java FragmentationTestClient localhost

On a LAN without packet loss, all messages succeed. But introduce artificial packet loss (using `tc` on Linux) or run this over a real network, and you will see the loss rate rise sharply once you exceed one MTU. Here is the equivalent in Go:

package main

import (
    "fmt"
    "net"
    "os"
    "strconv"
    "time"
)

func main() {
    host := "127.0.0.1"
    if len(os.Args) > 1 {
        host = os.Args[1]
    }
    port := 9999

    fmt.Printf("Testing UDP fragmentation to %s:%d\n", host, port)
    fmt.Println("Size\tFragments\tRecv\tLoss%%")

    for size := 100; size <= 8000; size += 500 {
        received := 0
        trials := 5

        for trial := 0; trial < trials; trial++ {
            if sendAndWait(host, port, size) {
                received++
            }
        }

        fragments := (size + 1471) / 1472
        loss := 100.0 * float64(trials-received) / float64(trials)

        fmt.Printf("%d\t%d\t\t%d\t%.0f%%\n", size, fragments, received, loss)
        time.Sleep(100 * time.Millisecond)
    }
}

func sendAndWait(host string, port int, size int) bool {
    addr := net.UDPAddr{
        Port: port,
        IP:   net.ParseIP(host),
    }

    conn, err := net.DialUDP("udp", nil, &addr)
    if err != nil {
        return false
    }
    defer conn.Close()

    conn.SetReadDeadline(time.Now().Add(1 * time.Second))

    request := fmt.Sprintf("SIZE:%d", size)
    _, err = conn.Write([]byte(request))
    if err != nil {
        return false
    }

    response := make([]byte, size)
    _, err = conn.Read(response)
    if err != nil {
        return false
    }

    return true
}

Compile and run the Go version:

go run main.go localhost

Both behave identically. On a clean link, both see 0% loss even at 8KB. But watch what happens if you simulate packet loss. On Linux:

sudo tc qdisc add dev lo root netem loss 1%

Now rerun the client. A 1% per-packet loss rate becomes devastating for fragmented datagrams. A single 5000-byte message fragments into 4 packets. The probability all 4 arrive is 0.99^4 = 0.96, so you lose 4% of 5000-byte messages. At 7000 bytes (5 fragments), you drop 5% of messages. This is not linear; it is exponential against fragment count.

Clean up the test:

sudo tc qdisc del dev lo root

Safe Limits and Best Practices

You have three options:

Approach	Max Payload	Fragmentation	Tradeoff
Conservative (DNS, NTP)	512 bytes	None (fits in ~1 packet)	Safe on any network; limits throughput
Standard Ethernet	1472 bytes	Single packet on local MTU 1500	Works on most LANs; fragmented over internet
Probe and adapt	Path MTU - 28	None on that path	Best throughput; requires PMTUD or manual discovery

For a production service, I recommend this pattern: send payloads under 512 bytes by default, expose a configuration knob for operators to set a larger limit (up to 1472) if they control the network, and implement application-level fragmentation if you need larger messages. That means you split the data yourself, send multiple small UDP packets, and reassemble on the receive side. This trades simplicity for reliability, but you control the retransmission strategy. You can retry failed fragments without waiting for the entire message to timeout.

Application-Level Fragmentation Example

Here is a simple message splitter in Java. The sender fragments a large message and sends each chunk with a sequence number. The receiver collects chunks and reassembles.

import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.nio.ByteBuffer;
import java.util.HashMap;
import java.util.Map;

public class ReliableUDPMessage {

  // Header format: 4 bytes magic + 2 bytes message ID + 2 bytes chunk index
  // + 2 bytes total chunks + payload
  static final int MAGIC = 0xDEADBEEF;
  static final int HEADER_SIZE = 10;
  static final int MAX_CHUNK_PAYLOAD = 1400; // Leave room for header

  static class MessageChunk {
    int messageId;
    int chunkIndex;
    int totalChunks;
    byte[] payload;
  }

  static class ReassemblyBuffer {
    byte[][] chunks;
    int totalChunks;
    int received = 0;

    ReassemblyBuffer(int totalChunks) {
      this.totalChunks = totalChunks;
      this.chunks = new byte[totalChunks][];
    }

    boolean add(int chunkIndex, byte[] payload) {
      if (chunks[chunkIndex] != null) {
        return false; // Already have this chunk
      }
      chunks[chunkIndex] = payload;
      received++;
      return received == totalChunks;
    }

    byte[] reassemble() {
      int totalSize = 0;
      for (byte[] chunk : chunks) {
        totalSize += chunk.length;
      }
      byte[] result = new byte[totalSize];
      int offset = 0;
      for (byte[] chunk : chunks) {
        System.arraycopy(chunk, 0, result, offset, chunk.length);
        offset += chunk.length;
      }
      return result;
    }
  }

  static MessageChunk decodeChunk(byte[] packet, int len) throws Exception {
    if (len < HEADER_SIZE) {
      throw new Exception("Packet too small");
    }

    ByteBuffer bb = ByteBuffer.wrap(packet, 0, len);
    int magic = bb.getInt();
    if (magic != MAGIC) {
      throw new Exception("Bad magic");
    }

    short messageId = bb.getShort();
    short chunkIndex = bb.getShort();
    short totalChunks = bb.getShort();

    int payloadLen = len - HEADER_SIZE;
    byte[] payload = new byte[payloadLen];
    bb.get(payload);

    MessageChunk chunk = new MessageChunk();
    chunk.messageId = messageId & 0xFFFF;
    chunk.chunkIndex = chunkIndex & 0xFFFF;
    chunk.totalChunks = totalChunks & 0xFFFF;
    chunk.payload = payload;
    return chunk;
  }

  static byte[] encodeChunk(int messageId, int chunkIndex, int totalChunks,
      byte[] payload) {
    byte[] packet = new byte[HEADER_SIZE + payload.length];
    ByteBuffer bb = ByteBuffer.wrap(packet);
    bb.putInt(MAGIC);
    bb.putShort((short) messageId);
    bb.putShort((short) chunkIndex);
    bb.putShort((short) totalChunks);
    bb.put(payload);
    return packet;
  }

  static class Sender {
    DatagramSocket socket;
    InetAddress targetAddr;
    int targetPort;
    int nextMessageId = 1;

    Sender(InetAddress addr, int port) throws Exception {
      this.socket = new DatagramSocket();
      this.targetAddr = addr;
      this.targetPort = port;
    }

    void sendMessage(byte[] message) throws Exception {
      int totalChunks = (message.length + MAX_CHUNK_PAYLOAD - 1)
          / MAX_CHUNK_PAYLOAD;
      int messageId = nextMessageId++;

      System.out.printf("Sending message %d (%d bytes in %d chunks)\n",
          messageId, message.length, totalChunks);

      for (int chunkIndex = 0; chunkIndex < totalChunks; chunkIndex++) {
        int offset = chunkIndex * MAX_CHUNK_PAYLOAD;
        int len = Math.min(MAX_CHUNK_PAYLOAD, message.length - offset);
        byte[] payload = new byte[len];
        System.arraycopy(message, offset, payload, 0, len);

        byte[] packet = encodeChunk(messageId, chunkIndex, totalChunks,
            payload);
        DatagramPacket dp = new DatagramPacket(packet, packet.length,
            targetAddr, targetPort);
        socket.send(dp);
        System.out.printf("  Chunk %d/%d sent (%d bytes)\n", chunkIndex + 1,
            totalChunks, packet.length);
      }
    }

    void close() {
      socket.close();
    }
  }

  static class Receiver {
    DatagramSocket socket;
    Map<Integer, ReassemblyBuffer> buffers = new HashMap<>();

    Receiver(int port) throws Exception {
      this.socket = new DatagramSocket(port);
    }

    byte[] receiveMessage(int timeoutMs) throws Exception {
      socket.setSoTimeout(timeoutMs);
      byte[] buffer = new byte[2048];
      DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

      while (true) {
        try {
          socket.receive(packet);
          MessageChunk chunk = decodeChunk(packet.getData(),
              packet.getLength());

          ReassemblyBuffer rb = buffers.computeIfAbsent(chunk.messageId,
              id -> new ReassemblyBuffer(chunk.totalChunks));

          if (rb.add(chunk.chunkIndex, chunk.payload)) {
            System.out.printf(
                "Message %d complete (%d chunks received)\n",
                chunk.messageId, chunk.totalChunks);
            buffers.remove(chunk.messageId);
            return rb.reassemble();
          }

          System.out.printf("Message %d chunk %d/%d received\n",
              chunk.messageId, chunk.chunkIndex + 1, chunk.totalChunks);
        } catch (java.net.SocketTimeoutException e) {
          throw new Exception("Timeout waiting for complete message");
        }
      }
    }

    void close() {
      socket.close();
    }
  }

  public static void main(String[] args) throws Exception {
    if (args.length == 0 || args[0].equals("server")) {
      // Server mode
      Receiver server = new Receiver(9998);
      System.out.println("Reliable UDP server on port 9998");
      while (true) {
        try {
          byte[] msg = server.receiveMessage(10000);
          System.out.printf("Received message: %s\n",
              new String(msg, 0, Math.min(msg.length, 100)));
        } catch (Exception e) {
          System.out.println("Error: " + e.getMessage());
        }
      }
    } else {
      // Client mode
      Sender client = new Sender(InetAddress.getByName("127.0.0.1"), 9998);

      // Test with a message larger than one MTU
      byte[] testMessage = new byte[5000];
      for (int i = 0; i < testMessage.length; i++) {
        testMessage[i] = (byte) ('A' + (i % 26));
      }

      client.sendMessage(testMessage);
      client.close();
    }
  }
}

Run the server in one terminal:

javac ReliableUDPMessage.java
java ReliableUDPMessage server

And send a message from another:

java ReliableUDPMessage client

This approach guarantees delivery of large messages because you control retransmission. If you know a chunk is missing, you can ask the sender to resend only that chunk, not the entire message. This is closer to how real protocols like QUIC handle it.

Observing Fragmentation with tcpdump

To see fragmentation in action, run tcpdump while sending large datagrams. On Linux:

sudo tcpdump -i lo -n 'udp port 9998' -X

Then send a 5000-byte message. You will see multiple IP packets, each with the same source/destination but different fragment offsets in the IP header. The More Fragments (MF) bit is set on all but the last, and the Fragment Offset field shows where each piece belongs in the original 5000-byte message.

Common Mistakes

Assuming IPv6 helps: IPv6 has a minimum MTU of 1280 bytes. That is worse, not better, in many scenarios. The effective payload is 1252 bytes (1280 - 20 for IPv6 header - 8 for UDP).

Ignoring the operating system send buffer: Many OSes apply a hard limit to UDP sends. On Linux, it is often 128 KB by default, but you can hit practical limits much sooner if fragmentation causes queueing. Check your SO_SNDBUF and SO_RCVBUF settings.

Testing only on localhost: Localhost does not fragment. It is a virtual interface. Always test over a real network or use a network simulator. The code examples above work on localhost, but real losses only show up under real conditions.

Trusting PMTUD in production: ICMP filtering is common in corporate environments. If you see sudden, inexplicable packet loss at certain sizes, PMTUD is likely broken. Hard-code a conservative limit (512 or 1472 bytes) and move on.

FAQ

What is the actual maximum UDP datagram size?

Theoretically 65,535 bytes (the UDP length field is 16 bits). Practically, 1472 bytes on most Ethernet links without fragmentation, and often lower on paths with tunnels or wireless. Anything beyond the path MTU minus 28 bytes (IP + UDP headers) will be fragmented. If any fragment is lost, the entire datagram is discarded.

Does TCP avoid this problem?

TCP uses stream segmentation, not fragmentation. The sender controls segment size (the MSS, Maximum Segment Size, negotiated during handshake). Both sides agree not to fragment. TCP also implements retransmission, so if one segment is lost, only that segment is resent, not the entire stream. UDP has neither luxury.

Is application-level fragmentation worth it?

Only if you need it. If your average message fits in 512 bytes, do not add it. If you are building a game server or media streaming protocol and messages routinely exceed 1472 bytes, yes, implement it or use a protocol that already does (QUIC, SCTP). The overhead is small: a 10-byte header per chunk plus the complexity of a reassembly buffer.

What if I set DF on all outgoing packets?

Good idea. Setting the Do-Not-Fragment bit forces the kernel to return an error rather than fragment silently. You can then catch that error and reduce the message size. In Java, this is not directly exposed in the DatagramSocket API, but you can do it via raw sockets or JNI. In Go, it is available as syscall options.

How do I measure packet loss in my UDP application?

Sequence numbers. Include a sequence number in every packet, and on the receive side, check for gaps. If you sent packets 1 through 100 and only received 1 through 95 and 97 through 100, you know packet 96 was lost. Log it, alert on it, or trigger a resend request. This is standard practice in real-time protocols.

Why does localhost not fragment?

The loopback device (lo) does not have a real MTU constraint. The kernel copies data directly into the receiver's buffer without going through a network interface. So fragmentation never happens. Always test UDP size limits on a real interface or use a network simulator like `netem`.

DEV Community