I'm trying to make a singleton using shared memory to make sure that the class object will be the same between all processes, but for this I need to serialize it:
class Singleton(type):
filename = path.basename(path.abspath(__file__)).split('.')[0]
def __call__(cls, *args, **kwargs):
instance = super(Singleton, cls).__call__(*args, **kwargs)
pickled = pickle.dumps(instance)
class SingleScheduler(APScheduler, metaclass=Singleton):
def __init__(self, *args, **kwargs):
super().__init__(args, kwargs)
TypeError: Schedulers cannot be serialized. Ensure that you are not passing a scheduler instance as an argument to a job, or scheduling an instance method where the instance contains a scheduler as an attribute.
Is there some way to serialize this?
P.S. Yes, I know that I can just take a redis and pass parameters and status to it, but I want to try to do it without any dependencies...
The scheduler was explicitly made unserializable because it would not serialize anyway because it contains synchronization primitives and references to the job store. A lot of people ran into pickling errors because of this so the 3.7.0 release added this explicit error message.
With enough effort, all of this could be worked around, with the exception of the memory job store. If your intention is to create multiple copies of the scheduler sharing a single external store, that is not going to work on APScheduler 3.x because it was not designed for distributed use. The 4.0 release will rectify this and it is the most prominent feature there. If, on the other hand, you were using a memory backed job store, serializing would just create multiple copies of the jobs, at which point you're better off creating a new scheduler for each process anyway.
If you want to share a scheduler between multiple processes, the way to do it on APScheduler 3.x is to have the scheduler run in a dedicated process and then communicate with said process over either RPyC, some web server or similar IPC or network mechanism.