Search code examples
pythondictionarythrift

How to avoid "TypeError: unhashable type" when Python Thrift decodes a map indexed by a struct


The file model.thrift contains the following Thrift model:

struct Coordinate {
    1: required i32 x;
    2: required i32 y;
}

struct Terrain {
    1: required map<Coordinate, i32> altitude_samples;
}

Note that we have a map (altitude_samples) indexed by a struct (Coordinate).

I use the Thrift compiler to generate Python encoding and decoding classes:

thrift -gen py model.thrift

I use the following Python code to decode a Terrain object from a file:

#!/usr/bin/env python

import sys
sys.path.append('gen-py')

import thrift.protocol.TBinaryProtocol
import thrift.transport.TTransport
import model.ttypes

def decode_terrain_from_file():
    file = open("terrain.dat", "rb")
    transport = thrift.transport.TTransport.TFileObjectTransport(file)
    protocol = thrift.protocol.TBinaryProtocol.TBinaryProtocol(transport)
    terrain = model.ttypes.Terrain()
    terrain.read(protocol)
    print(terrain)

if __name__ == "__main__":
    decode_terrain_from_file()

When I run this program, I get the following error:

(env) $ python py_decode.py 
Traceback (most recent call last):
  File "py_decode.py", line 19, in <module>
    decode_terrain_from_file()
  File "py_decode.py", line 15, in decode_terrain_from_file
    terrain.read(protocol)
  File "gen-py/model/ttypes.py", line 119, in read
    self.altitude_samples[_key5] = _val6
TypeError: unhashable type: 'Coordinate

Solution

  • The problem is that the Thrift compiler cannot automatically generate the hash function for the Coordinate class.

    You must add the hash function manually as follows:

    #!/usr/bin/env python
    
    import sys
    sys.path.append('gen-py')
    
    import thrift.protocol.TBinaryProtocol
    import thrift.transport.TTransport
    import model.ttypes
    
    model.ttypes.Coordinate.__hash__ = lambda self: hash((self.x, self.y))
    
    def decode_terrain_from_file():
        file = open("terrain.dat", "rb")
        transport = thrift.transport.TTransport.TFileObjectTransport(file)
        protocol = thrift.protocol.TBinaryProtocol.TBinaryProtocol(transport)
        terrain = model.ttypes.Terrain()
        terrain.read(protocol)
        print(terrain)
    
    if __name__ == "__main__":
        decode_terrain_from_file()
    

    Note that a similar problem occurs for C++ generated code. In the C++ case you need to manually add the Coordinate::operator< to the program. Thrift does generate the declaration for the Coordinate::operator< but does not generate the implementation of Coordinate::operator<. Once again, the reason for this is that Thrift does not understand the semantics of the struct and hence cannot guess the correct implementation of the comparison operator.

    bool Coordinate::operator<(const Coordinate& other) const
    {
        if (x < other.x) {
            return true;
        } else if (x > other.x) {
            return false;
        } else if (y < other.y) {
            return true;
        } else {
            return false;
        }
    }