Accessing values in a dict by key in multiprocessing

Question

Using the multiprocessing module, I wrote a server to serve a dict. Now, trying to access that dict by key with a client, I get the following error (server.py and client.py are at the bottom of the post):

Traceback (most recent call last):
  File "client.py", line 19, in <module>
    item = my_dict[key]
TypeError: 'AutoProxy[get_dict]' object is not subscriptable

I believe this is due to the fact that the dict that I register()-ed with my SyncManager gets pickled and passed on as an AutoProxy object. When I check the methods of the AutoProxy object with print(dir(my_dict)), this is what I get:

['_Client', '__builtins__', '__class__', '__deepcopy__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_address_to_local', '_after_fork', '_authkey', '_callmethod', '_close', '_connect', '_decref', '_exposed_', '_getvalue', '_id', '_idset', '_incref', '_isauto', '_manager', '_mutex', '_serializer', '_tls', '_token', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']

Which is different from the output of print(dir(my_dict)) in server.py:

['__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__init__', '__iter__', '__le__', '__len__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'clear', 'copy', 'fromkeys', 'get', 'items', 'keys', 'pop', 'popitem', 'setdefault', 'update', 'values']

It looks like this Autoproxy object retains some methods of the dict object, but apparently not all of them. Crucially it doesn't retain the __getitem__ method, which prevents me from accessing items by key.

How can I access the dict items by key? Also any explanation of how Proxys work with Python multiprocessing would be very helpful.

Note: I don't need to modify the dict values, I just need to extract them by key.

server.py:

from multiprocessing.managers import SyncManager

my_dict = {'item_1': 1, 'item_2':2}
def get_my_dict():
    return my_dict

class MyManager(SyncManager):
    pass

if __name__ == "__main__":
    port_num = 4343
    MyManager.register("get_dict", get_my_dict)
    manager = MyManager(("127.0.0.1", port_num), authkey=b"password")
    manager.start()
    input("Press any key to kill server".center(50, "-"))
    manager.shutdown

client.py

from multiprocessing.managers import SyncManager
import sys

class MyManager(SyncManager):
    pass

MyManager.register("get_dict")

if __name__ == "__main__":
    port_num = 4343
    manager = MyManager(("127.0.0.1", port_num), authkey=b"password")
    manager.connect()
    my_dict = manager.get_dict()

    print("dict = %s" % (dir(my_dict)))
    keys = list(my_dict.keys())
    print(keys)
    for key in keys:
        print(my_dict[key])

Answer 1

You can use get to, well, get an element of a dictionary, in either version.