Search code examples
pythonserializationpickleuse-case

Common use-cases for pickle in Python


I've looked at the pickle documentation, but I don't understand where pickle is useful.

What are some common use-cases for pickle?


Solution

  • Some uses that I have come across:

    1) saving a program's state data to disk so that it can carry on where it left off when restarted (persistence)

    2) sending python data over a TCP connection in a multi-core or distributed system (marshalling)

    3) storing python objects in a database

    4) converting an arbitrary python object to a string so that it can be used as a dictionary key (e.g. for caching & memoization).

    There are some issues with the last one - two identical objects can be pickled and result in different strings - or even the same object pickled twice can have different representations. This is because the pickle can include reference count information.

    To emphasise @lunaryorn's comment - you should never unpickle a string from an untrusted source, since a carefully crafted pickle could execute arbitrary code on your system. For example see https://blog.nelhage.com/2011/03/exploiting-pickle/