TL;DR: Interested in knowing if it's possible to use Abstract Base Classes as a mixin in the way I'd like to, or if my approach is fundamentally misguided.
I have a Flask project I've been working on. As part of my project, I've implemented a RememberingDict class. It's a simple subclass of dict, with a handful of extra features tacked on: it remembers its creation time, it knows how to pickle/save itself to a disk, and it knows how to open/unpickle itself from a disk:
from __future__ import annotations
import pickle
from datetime import datetime
from typing import Final, Optional, TypeVar, Any, Hashable
FILE_PATH: Final = 'data.pickle'
T = TypeVar('T', bound='RememberingDict')
class RememberingDict(dict):
def __init__(self, data: Optional[dict[Hashable, Any]] = None) -> None:
super().__init__(data if data is not None else {})
self.creation_time: datetime = datetime.now()
def to_disk(self) -> None:
"""I save a copy of the data to a file"""
with open(FILE_PATH, 'wb') as f:
pickle.dump(self, f)
@classmethod
def from_disk(cls: type[T]) -> T:
"""I extract a copy of the data from a file"""
with open(FILE_PATH, 'rb') as f:
latest_dataset: T = pickle.load(f)
return latest_dataset
The code works really nicely for my purposes while on a local development server, so all is good, but (for reasons it is unnecessary here to go into), it doesn't work when deploying it on Google App Engine, so for those purposes, I designed this alternative implementation:
from __future__ import annotations
import pickle
from datetime import datetime
from typing import Optional, TypeVar, Hashable, Any
from google.cloud.storage.blob import Blob
def get_google_blob() -> Blob:
"""
Actual implementation unnecessary to go into,
but rest assured that the real version of this function returns a Blob object,
linked to Google Storage account credentials,
from which files can be uploaded to, and downloaded from,
Google's Cloud Storage platform.
"""
pass
T = TypeVar('T', bound='RememberingDict')
class RememberingDict(dict):
def __init__(self, data: Optional[dict[Hashable, Any]] = None) -> None:
super().__init__(data if data is not None else {})
self.creation_time: datetime = datetime.now()
def to_disk(self) -> None:
"""I upload a copy of the data to Google's Cloud Storage"""
get_google_blob().upload_from_string(pickle.dumps(self))
@classmethod
def from_disk(cls: type[T]) -> T:
"""I extract a copy of the data from Google's Cloud Storage"""
latest dataset: T = pickle.loads(get_google_blob().download_as_bytes())
return latest_dataset
Now, both of these implementations work fine. However, I want to keep them both -- the first one is useful for development -- but the annoying thing is that there's obviously a fair amount of repetition between the two. Their __init__() functions are identical; they both have a to_disk() method that saves the instance to a file and returns None; and they both have a from_disk() classmethod that returns an instance of the class that's been saved to a disk somewhere.
Ideally, I would like to have them both inherit from a base class, which passes them a variety of dict-like abilities, and also specifies that the to_disk() and from_disk() methods must be overridden in order to provide a complete implementation.
This feels like a problem that ABCs should be able to solve. I tried the following:
from __future__ import annotations
from datetime import datetime
from typing import Final, Optional, TypeVar, Hashable, Any
from abc import ABC, abstractmethod
from google.cloud.storage.blob import Blob
T = TypeVar('T', bound='AbstractRememberingDict')
class AbstractRememberingDict(ABC, dict):
def __init__(self, data: Optional[dict[Hashable, Any]] = None) -> None:
super().__init__(data if data is not None else {})
self.creation_time: datetime = datetime.now()
@abstractmethod
def to_disk(self) -> None: ...
@classmethod
@abstractmethod
def from_disk(cls: type[T]) -> T: ...
FILE_PATH: Final = 'data.pickle'
class LocalRememberingDict(AbstractRememberingDict):
def to_disk(self) -> None:
"""I save a copy of the data to a file"""
with open(FILE_PATH, 'wb') as f:
pickle.dump(self, f)
@classmethod
def from_disk(cls: type[T]) -> T:
"""I extract a copy of the data from a file"""
with open(FILE_PATH, 'rb') as f:
latest_dataset: T = pickle.load(f)
return latest_dataset
def get_google_blob() -> Blob:
"""
Actual implementation unnecessary to go into,
but rest assured that the real version of this function returns a Blob object,
linked to Google Storage account credentials,
from which files can be uploaded to, and downloaded from,
Google's Cloud Storage platform.
"""
pass
class RemoteRememberingDict(AbstractRememberingDict):
def to_disk(self) -> None:
"""I upload a copy of the data to Google's Cloud Storage"""
get_google_blob().upload_from_string(pickle.dumps(self))
@classmethod
def from_disk(cls: type[T]) -> T:
"""I extract a copy of the data from Google's Cloud Storage"""
latest_dataset: T = pickle.loads(get_google_blob().download_as_bytes())
return latest_dataset
However, using the ABC as a mixin (rather than as the sole base class) appears to mess with the @abstractmethod decorator, such that inherited classes no longer raise an exception if they fail to implement the required abstract methods.
Ideally, I'd like my base class to inherit all the features of a standard Python dict, but also specify that certain methods must be implemented in inherited classes for instances to be instantiated.
Is what I'm trying to do possible, or is my approach fundamentally misguided?
(As an aside: I'm more interested in the way ABCs work than about the best way to cache data structures for a web app, etc. -- I'm sure there may be better ways of caching data, but this is my first Flask project and my way's working great for me at the moment.)
You can get around the problems of subclassing dict by subclassing collections.UserDict instead. As the docs say:
Class that simulates a dictionary. The instance’s contents are kept in a regular dictionary, which is accessible via the data attribute of UserDict instances. If initialdata is provided, data is initialized with its contents; note that a reference to initialdata will not be kept, allowing it be used for other purposes.
Essentially, it's a thin regular-class wrapper around a dict. You should be able to use it with multiple inheritance as an abstract base class, as you do with AbstractRememberingDict.