pyignite serialize and de serialize complex types

Question

I'm trying to serialize and deserialize a json map(Map complex type in pyignite). The serialize seems to work ok but I'm getting an error in the deserialize. I'm not sure if my serialize is correct but it dumps out an array of ints that looks like a binary byte array.

Traceback (most recent call last):
File "ignite-sql.py", line 44, in <module>
  print([[result[0], result[1], result[2], Map.from_python(result[3])] 
  for result in results])
File "ignite-sql.py", line 44, in <listcomp>
  print([[result[0], result[1], result[2], Map.from_python(result[3])] 
  for result in results])
File "env/lib/python3.7/site-packages/pyignite/datatypes/complex.py", line 276, in from_python
  for k, v in value.items():
AttributeError: 'list' object has no attribute 'items'

Here is minimal example

client = Client()
client.connect('127.0.0.1', 10800)

PRESENCE_TABLE = '''
CREATE TABLE IF NOT EXISTS presence (
  subkey VARCHAR,
  channel VARCHAR,
  uuid VARCHAR,
  metadata BINARY,
  PRIMARY KEY (subkey, channel, uuid)
)'''

DROP_PRESENCE_TABLE = '''
DROP TABLE IF EXISTS presence
'''

INSERT_PRESENCE_TABLE = '''
INSERT INTO presence(
  subkey, channel, uuid, metadata
  ) VALUES (?, ?, ?, ?)
  '''


obj = Map.from_python({"foo":"bar"})
args = ["test","foo",str(uuid.uuid4()),obj]
client.sql(DROP_PRESENCE_TABLE)
client.sql(PRESENCE_TABLE)
client.sql(INSERT_PRESENCE_TABLE,query_args=args)

results = client.sql('select * from presence')

print([[result[0], result[1], result[2], Map.from_python(result[3])] for result in results])

Answer 1

I am an author and maintainer of pyignite.

I will try to answer your question, but since I'm not allowed to comment on questions here, I will have to make some guesses about what you try to do with your code.

1. It seems like you want to put a map object into the SQL database. Unfortunately, Ignite does not work this way. The data types that Ignite SQL uses are described here. No maps or other complex data types, as you can see.

I am not sure what you try to achieve, but may be you could use a foreign key on another SQL table for that purpose? It makes more sense for me than storing hash table in SQL column. Alternatively, you can JSONize your data and store it as String (VARCHAR).

2. Another thing is how you use from_python() method. These methods are for internal use only. They creates byte sequences that is useless from the user standpoint.

The first thought behind pyignite API was that user should be able to use only builtin Python types to store and retrieve data in Ignite cluster. All serialization is under the hood. No serializer methods or objects are available. Simple examples were written to give you this idea.

3. Class Map is undocumented and should not be used. The MapObject class can be used as a type hint in that rare cases, when just giving pyignite a dict object is ambiguous. Please read this section of the docs about how to use type hints.

Saying all that, I hope I was able to point you in a right direction. If I was not, please try to clarify your question, and I will improve my answer.

Good luck and thank you for using pyignite.