Search code examples
pythonvisual-studio-codepython-typingpython-3.11pyright

What is the correct way to please the typechecker for a '(bytes | str) -> str' function?


I have the following code:

def from_utf8(string: bytes | str) -> str:
    if isinstance(string, bytes):
        return string.decode("utf-8")
    else:
        return string  # <- type warning on this line

pylance gives me a type warning on the return string line:

Type "bytearray | memoryview[_I@memoryview] | str" is not assignable to return type "str"
  Type "bytearray | memoryview[_I@memoryview] | str" is not assignable to type "str"
    "bytearray" is not assignable to "str"

My understanding is:
the type annotation x: bytes is actually an alias for "runtime types" x: bytes | bytearray | memoryview[_I@memoryview], but isinstance(x, bytes) only checks for bytes, not the two others.

I tried checking for types the other way around:

def from_utf8(string: bytes | str) -> str:
    if isinstance(string, str):
        return string
    else:
        return string.decode("utf-8")  # <- no attribute 'decode' for 'memoryview'

The error now becomes:

Cannot access attribute "decode" for class "memoryview[_I@memoryview]"
  Attribute "decode" is unknown

For context:

  • my project uses python 3.11
  • I see these warnings in vscode, using pylance version 2025.2.1 and python (ms-python.python) extension version 2025.0.0

Do I have a convenient way to write a version of from_utf8(string) that passes the type checker ?

also: is my assumption correct, and is it documented somewhere ?


Solution

  • Before Python 3.12, bytes was specified to behave as an alias of builtins.bytes | builtins.bytearray | builtins.memoryview. From the Python 3.10 docs (emphasis mine):

    class typing.ByteString(Sequence[int])

    A generic version of collections.abc.ByteString.

    This type represents the types bytes, bytearray, and memoryview of byte sequences.

    As a shorthand for this type, bytes can be used to annotate arguments of any of the types mentioned above.

    The static typing error you're seeing is a consequence of this behaviour. This behaviour is now removed in Python 3.12 with the introduction of PEP 688.

    pyright 1.1.329 (released over a year ago) has since disabled this static typing behaviour by default under strict mode. If you don't want to use strict mode but still want to disable this behaviour, set disableBytesTypePromotions to true.


    As pointed out in the comments, a typed third party library may have been developed under this behaviour, in which case you should watch out when referring to variables or return values of functions from this library. As an example, without the --strict-bytes option, mypy will pass the following in type-checking (see mypy Playground):

    def f() -> bytes:
        return memoryview(b"")