Search code examples
postgresqldatabase-design

What is the recommended collation for a PostgreSQL unique e-mail column?


Most places recommend for unique e-mail indexes using either a lower(email) index, or a datatype citext.

But the manual for citext suggests using a collation for all cases instead of citext now:

Consider using nondeterministic collations instead of this module.

So, currently, what is the recommended way to treat a unique e-mail column in PostgreSQL?

Index on lower()? citext still? Or if collation, which one exactly?


Solution

  • if collate, which one exactly?

    Whichever one you prefer - it's not necessarily about picking a specific collation out of the built-in list, but configuring one that way. As demonstrated in the doc:
    demo at db<>fiddle

    CREATE COLLATION case_insensitive (
      provider = icu, 
      locale = 'und-u-ks-level2', --undefined(root,generic), but you can pick a language
      deterministic = false);
    SELECT 'a' = 'A' COLLATE case_insensitive; -- true
    

    If you're asking about which locale is recommended for the case-insensitive collation, I'd say that's the generic one above but ultimately it won't matter. Others mainly differ in how they sort things, and unless specifically configured for the purpose, I don't think they'll disagree on equality you want for unique.

    I assumed you're asking which particular collation is the recommendation, in which case there's no such thing - a standard installation doesn't ship with a built-in case-insensitive collation out of the box. Japanese, Arabic and Chinese are examples of "case-insensitive" languages (characters don't have upper/lower variants) but collations based on them aren't themselves case-insensitive when they deal with other alphabets.

    If you're asking about performance, in this test on 150k 90% unique random samples as well as another one I ran on 10M, the expression index using lower() on a text column with collate "C" tends to perform best, followed closely by a regular index on the text column using the case-insensitive icu collation. citext stays way behind.


    My point is that the character set for email is not open to interpretation. email rfc uses ascii, with several chars removed, it doesn't care about internationalization standards.

    There's more than one "email RFC" so you could argue that even the meaning of "email RFC" is open to interpretation. Also, it is not "ASCII-only, with several chars removed", or at least it hasn't been for a while now. Here's a relatively old thread discussing these topics, linking a nice example at Wikipedia:

    The example addresses below would not be handled by RRFC 5322 based servers, but are permitted by RFC 6530. Servers compliant with this will be able to handle these:

    • Latin alphabet with diacritics: Pelé@example.com
    • Greek alphabet: δοκιμή@παράδειγμα.δοκιμή
    • Traditional Chinese characters: 我買@屋企.香港
    • Japanese characters: 二ノ宮@黒川.日本
    • Cyrillic characters: медведь@с-балалайкой.рф
    • Devanagari characters: संपर्क@डाटामेल.भारत

    I feel like I need to draw a line: character sets are about encoding. Collations you asked about are for sorting and equality, and most work with all encoding options. You can tell by checking pg_collation.collencoding = -1:

    collencoding int4
    Encoding in which the collation is applicable, or -1 if it works for any encoding

    In case you wanted to ask about email address validation and normalisation through a column constraint, that's a slightly different question.