Serguey Asael Shinder: Two strings that look identical and are not equal: Unicode normalisation in Java

20 September 2026, 10:40 · by Serguey Asael Shinder / Serguey Shinder

Here is a bug that takes an afternoon the first time and five minutes forever after. A user uploads a file called résumé.pdf. Your code stores the name, later looks it up, and the lookup fails. The string in the log is résumé.pdf. The string in the map is résumé.pdf. They are not equal.

The reason is that Unicode offers more than one way to write the same visible text. The é can be a single code point, U+00E9, or it can be two: a plain e followed by U+0301, the combining acute accent. Both render identically in every font you are likely to use. String.equals compares sequences of char values, so it says they are different — correctly, because they are.

String composed   = "résumé";   // é as one code point
String decomposed = "résumé"; // e + combining acute
composed.equals(decomposed);   // false
composed.length();             // 6
decomposed.length();           // 8

Where the two forms come from

You do not get to choose which form arrives. macOS has historically stored filenames in a decomposed form, so a name that travelled through a Mac filesystem often arrives as e plus an accent. Windows and most Linux tooling produce the composed form. The same is true of text pasted from different applications, of data loaded from systems with different normalisation policies, and of anything that has been through a round of transliteration.

The consequence is not limited to equals. hashCode differs, so HashMap and HashSet lookups miss. Sorting puts the two forms apart. A UNIQUE constraint in a database will happily accept both, and then you have two rows that look like duplicates to every human who reads the table and like distinct values to the engine.

The fix

Normalise at the boundary, once, to a form you choose, and keep everything inside that boundary in that form:

import java.text.Normalizer;
static String canonical(String s) {
    return Normalizer.normalize(s, Normalizer.Form.NFC);
}

NFC — composed — is the sensible default for stored text: it is what the web platform recommends, it is shorter, and it is what most sources already produce. NFD is useful when you want to strip accents, because once the marks are separate code points you can remove them with a single filter, but that is a transformation, not a storage format.

Normalizer.isNormalized lets you check without allocating a new string, which matters if you are doing this on a hot path.

The three rules I use

Normalise on input, not on comparison. A canonical() call scattered through comparison sites will be forgotten in exactly one place, and that place will be the one with the user-visible bug. Do it where data enters the system: request parsing, file ingestion, the row mapper.

Store the normalised form, and say so. If the database holds NFC, write it in the schema documentation next to the column. Otherwise the next person will add an index and wonder why two rows with the same name do not collide.

Do not use normalisation for case or locale. Normalizer solves the encoding question only. For case-insensitive matching you still need equalsIgnoreCase or a locale-aware Collator, and the Turkish dotless ı will still ruin your afternoon if you apply toLowerCase() without a locale. Those are separate problems that arrive in the same bug report.

The general lesson is older than Unicode: when two representations of one value can both exist, pick one at the door and never let the other inside. Filenames, email addresses, phone numbers and URLs all have this shape, and the failure looks the same every time — a comparison that is correct about bytes and wrong about meaning.