Search code examples
pythonmysqlgmpgmpy

Storing really huge ints in MySQL

Is it possible to efficiently store big ints in MySQL? Not big as in bigint, more like 1000th Catalan number or so, and there are at least hundreds of them. More likely, thousands. The speed of storing and loading values is a priority.

The numbers are created using gmpy2, Python's GMP wrapper. So it'd be nice if the stored numbers were accessible in the same format, but if not, I can always write a converter.

Solution

  • I would convert the mpz integer to a Python string (for v2) or bytes (for v3) and save it as a BLOB entity in mysql. Here are the relevant gmpy2 commands using Python 2.

    >>> import gmpy2
>>> gmpy2.mpz(1234)
mpz(1234)
>>> gmpy2.to_binary(mpz(1234))
'\x01\x01\xd2\x04'
>>> gmpy2.from_binary('\x01\x01\xd2\x04')
mpz(1234)
>>>

    gmpy2.to_binary will convert an arbitrary gmpy2 object to a binary string. gmpy2.from_binary will convert the binary string back to a gmpy2 object. The object's type is encoded into the binary string so you don't need to track the object's type.

    Disclaimer: I maintain gmpy2.