Can the Django ORM store an unsigned 64-bit integer (aka ulong64 or uint64) in a reliably backend-agnostic manner?

Question

All the docs I've seen imply that you might be able to do that, but there isn't anything official w/r/t ulong64/uint64 fields. There are a few off-the-shelf options that look quite promising in this arena:

• BigIntegerField ... almost, but signed;
• PositiveIntegerField ... suspiciously 32-bit-looking; and
• DecimalField ... a fixed-pointer represented with a python decimal type, according to the docs -- which presumably turns into an analogously pedantic and slow database field when socked away, á la the DECIMAL or NUMERIC PostgreSQL types.

... all of which look like they might store a number like that. Except NONE OF THEM WILL COMMIT, much like every single rom-com character portrayed by Hugh Grant.

My primary criterion is that it works with Django's supported backends, without any if postgresql (...) elif mysql (...) type of special-case nonsense. After that, there is the need for speed -- this is for a model field in an visual-database application that will index image-derived data (e.g. perceptual hashes and extracted keypoint features), allowing ordering and grouping by the content of those images.

So: is there a good Django extension or app that furnishes some kind of PositiveBigIntegerField that will suit my purposes?

And, barring that: If there is a simple and reliable way to use Django's stock ORM to store unsigned 64-bit ints, I'd like to know it. Look, I'm no binary whiz; I have to do two's complement on paper -- so if this method of yours involves some bit-shifting trickery, don't hesitate to explain what it is, even if it strikes you as obvious. Thanks in advance.

Answer 1

Although I did not test it, but you may wish to just subclass BigIntegerField. The original BigIntegerField looks like that (source here):

class BigIntegerField(IntegerField):
    empty_strings_allowed = False
    description = _("Big (8 byte) integer")
    MAX_BIGINT = 9223372036854775807

    def get_internal_type(self):
        return "BigIntegerField"

    def formfield(self, **kwargs):
        defaults = {'min_value': -BigIntegerField.MAX_BIGINT - 1,
                    'max_value': BigIntegerField.MAX_BIGINT}
        defaults.update(kwargs)
        return super(BigIntegerField, self).formfield(**defaults)

Derived PositiveBigIntegerField may looks like this:

class PositiveBigIntegerField(BigIntegerField):
    empty_strings_allowed = False
    description = _("Big (8 byte) positive integer")

    def db_type(self, connection):
        """
        Returns MySQL-specific column data type. Make additional checks
        to support other backends.
        """
        return 'bigint UNSIGNED'

    def formfield(self, **kwargs):
        defaults = {'min_value': 0,
                    'max_value': BigIntegerField.MAX_BIGINT * 2 - 1}
        defaults.update(kwargs)
        return super(PositiveBigIntegerField, self).formfield(**defaults)

Although you should test it thoroughly, before using it. If you do, please share the results :)

EDIT:

I missed one thing - internal database representation. This is based on value returned by get_internal_type() and the definition of the column type is stored eg. here in case of MySQL backend and determined here. It looks like overwriting db_type() will give you control over how the field is represented in the database. However, you will need to find a way to return DBMS-specific value in db_type() by checking connection argument.