I was trying to write a metaclass named Singleton, that, of course, implement the singleton design pattern:
class Singleton(type):
def __new__(cls, name, bases = None, attrs = None):
if bases is None:
bases = ()
if attrs is None:
attrs = {}
new_class = type.__new__(cls, name, bases, attrs)
new_class._instance = None
return new_class
def __call__(cls, *args, **kwargs):
if cls._instance is None:
cls._instance = cls.__new__(cls, *args, **kwargs)
cls.__init__(cls._instance, *args, **kwargs)
return cls._instance
This seems to work correctly:
class Foo(metaclass = Singleton):
pass
foo1 = Foo()
foo2 = Foo()
print(foo1 is foo2) # True
However, PyCharm gave me this warning for cls._instance = cls.__new__(cls, *args, **kwargs):
Expected type 'Type[Singleton]', got 'Singleton' instead
...and this for cls.__init__(cls._instance, *args, **kwargs):
Expected type 'str', got 'Singleton' instead
I ran mypy on the same file, and here's what it said:
# mypy test.py
Success: no issues found in 1 source file
I'm using Python 3.11, PyCharm 2023.1.1 and mypy 1.3.0 if that makes a difference.
So what exactly is the problem here? Am I doing this correctly? Is this a bug with PyCharm, with mypy or something else? If the error is on me, how can I fix it?
Since this is an XY Problem, I'll start with the solution to X. The answers for Y are further down.
There is no need for manually calling cls.__new__ and then cls.__init__ in Singleton.__call__. You can just call super().__call__ instead, just like @Grismar did in his answer.
There is also no need for a custom Singleton.__new__ method at all, if all you want to do is set up the singleton pattern in a type safe manner.
And to have a _instance = None fallback in your classes, you can just define and assign that attribute on the meta class.
Here is the minimal setup required:
from __future__ import annotations
from typing import Any
class Singleton(type):
_instance: Singleton | None = None
def __call__(cls, *args: Any, **kwargs: Any) -> Singleton:
if cls._instance is None:
cls._instance = super().__call__(*args, **kwargs)
return cls._instance
class Foo(metaclass=Singleton):
pass
This passes mypy --strict and causes no PyCharm warnings. You said you wanted _instance to be saved in each class not in the meta class. This is the case here. Try the following:
foo1 = Foo()
foo2 = Foo()
print(foo1 is foo2) # True
print(foo1 is Foo._instance) # True
print(Singleton._instance) # None
If you do want a custom meta class __new__ method, it requires a lot more boilerplate to set up in a type safe manner, due to the overloaded type constructor. But here is a template that should work in all situations:
from __future__ import annotations
from typing import Any, TypeVar, overload
T = TypeVar("T", bound=type)
class Singleton(type):
_instance: Singleton | None = None
@overload
def __new__(mcs, o: object, /) -> type: ...
@overload
def __new__(
mcs: type[T],
name: str,
bases: tuple[type, ...],
namespace: dict[str, Any],
/,
**kwargs: Any,
) -> T: ...
def __new__(
mcs,
name: Any,
bases: Any = (),
namespace: Any = None,
/,
**kwargs: Any,
) -> type:
if namespace is None:
namespace = {}
# do other things here...
return type.__new__(mcs, name, bases, namespace, **kwargs)
def __call__(cls, *args: Any, **kwargs: Any) -> Singleton:
if cls._instance is None:
cls._instance = super().__call__(*args, **kwargs)
return cls._instance
The __new__ overloads closely resemble those in the typeshed stubs.
But again, that is not necessary for the singleton-via-metaclass pattern.
Digging deeper into your type inference questions sent me down a rabbit hole, so the explanations got a little longer. But I thought it more useful to present the solution to your actual problem up front (because it circumvents the errors). So if you are interested in the explanations, keep reading.
cls.__new__ expect type[Singleton]?I get the same error from Mypy for your code:
Argument 1 to "__new__" of "Singleton" has incompatible type "Singleton"; expected "Type[Singleton]"
Recall that the __call__ method is an instance method. In the context of Singleton.__call__, the type of the first argument (in this case named cls) is inferred as Singleton.
Since you defined your own Singleton.__new__ method, its (implicit) signature will be reflected in cls.__new__. You did not annotate Singleton.__new__, but type checkers typically fall back to "standard" inferences for special cases like the first parameter of a method.
__new__ is a static method that takes a class as the first argument and returns an instance of it. The first argument to Singleton.__new__ is therefore expected to be the type type[Singleton], not an instance of Singleton.
So from the point of view of the type checker, by calling cls.__new__(cls, ...) you are passing an instance of Singleton as an argument, where a the type Singleton itself (or a subtype) is expected.
Side note:
This distinction can be quite confusing, which is one of the reasons why it is best practice to name the first parameter to __new__ differently than the first parameter to a "normal" method.
In regular classes (not inheriting from type) the first parameter to normal methods should be called self, while the first parameter to __new__ should be called cls.
In meta classes however, the conventions are not as ubiquitous, but common sense suggests that the normal methods' first parameter should be called cls, while the first parameter to __new__ should be called mcs (or mcls or something like that). It is just very useful to highlight the distinction in the nature of that first argument. But these are all conventions of course and the interpreter doesn't care either way.
Whether or not this inference of cls.__new__ as Singleton.__new__ is justified or sensible is debatable.
Since cls in that context will always be an instance of the surrounding (meta) class, I would argue that it does not make sense to expect cls.__new__ to ever resolve to the __new__ method of said (meta) class.
In fact, unless the class cls itself defines a custom __new__ method, it will fall back to object.__new__, not to Singleton.__new__:
class Singleton(type):
def __call__(cls, *args, **kwargs):
print(cls.__new__)
print(cls.__new__ is object.__new__)
print(cls.__new__ is Singleton.__new__)
class Foo(metaclass=Singleton):
pass
foo1 = Foo()
Output:
<built-in method __new__ of type object at 0x...>
True
False
object.__new__ indeed does accept Singleton as its first argument because it is a class and the return type would be an instance of it. So there is nothing wrong or unsafe about the way you called cls.__new__ as far as I can tell.
We can see the wrong type inference by the type checker even more clearly, if we add a custom __new__ to Singleton and run it through the type checker:
from __future__ import annotations
from typing import Any
class Singleton(type):
def __new__(mcs, name: str, bases: Any = None, attrs: Any = None) -> Singleton:
return type.__new__(mcs, name, bases, attrs)
def __call__(cls, *args: Any, **kwargs: Any) -> Any:
reveal_type(cls.__new__)
...
Mypy wrongly infers the type as follows:
Revealed type is "def (mcs: Type[Singleton], name: builtins.str, bases: Any =, attrs: Any =) -> Singleton"
So it clearly expects cls.__new__ to be Singleton.__new__, even though it is actually object.__new__.
As far as I understand it, the discrepancy between the actual method resolution and the one inferred by the type checker is due to the special-cased behavior for the __new__ method. It may also just have to do with meta classes being poorly supported by type checkers. But maybe someone more knowledgeable can clear this up. (Or I'll consult the issue tracker.)
__init__ callThe PyCharm message is nonsense of course. The problem seems to come down to the same faulty inference of cls.__init__ as type.__init__, as opposed to object.__init__.
Mypy has an entirely different problem, complaining about the explicit usage of __init__ on an instance with the following error:
Accessing "__init__" on an instance is unsound, since instance.__init__ could be from an incompatible subclass
The __init__ method is intentionally excluded from LSP conformity requirements by mypy, which means that explicitly calling it is technically unsafe.
Not much else to say. Avoid that call, unless you are sure the overrides all the way up the MRO chain for __init__ are type safe; then use # type: ignore[misc].
So to summarize, I believe these two warnings/errors are both false positives.