Text / byte checks before handoff

Split long text at a UTF-8 byte ceiling without loss

Character count alone does not prove that a receiver's byte limit will be met. Keep Unicode code points whole, record every chunk's byte count and verify the rejoined result before delivery.

Characters are not bytes

A🙂B looks like three characters but is 6 UTF-8 bytes. A JavaScript length check alone can cross a receiver limit when Japanese text or emoji appears near a boundary.

fixturelimitsource byteschunk bytesrejoin
ASCII464 | 2equal
Japanese7156 | 6 | 3equal
emoji565 | 1equal
boundary461 | 4 | 1equal

Keep code points whole

Add the next Unicode code point while the encoded chunk remains at or below the ceiling. If adding it would exceed the ceiling, move it to the next chunk; never delete, replace or cut a code point in half.

const bytes = new TextEncoder().encode(part).length;
if (bytes <= limit) keep(part); else nextChunk();

The receiver—an API, queue, database or file format—owns the actual limit and encoding.

Verify before delivery

  1. Confirm the receiver's encoding and byte ceiling.
  2. Record UTF-8 bytes for every chunk.
  3. Join chunks in their original order.
  4. Compare the joined result with the original source.
  5. Stop and review any over-limit or unknown contract.

This guide is static browser-local material. Input, files and verification results are not sent to or stored by an external service.

Do not treat an over-limit chunk as success

With a 4-byte ceiling, 🙂B is 5 bytes and cannot be delivered as one valid chunk. Split again or ask the receiver; do not silently truncate or omit data.