I'm building PyPi package, which is compatible with Python 3.8 and higher. That means that the minimum version of Python is 3.8, so I build the package in Python 3.8 environment.
My question is should I build package separately for Python 3.8 and 3.9?
I think the thing that actually builds package is the module, build, so the version of Python doesn't matter. Isn't it?
The version does not matter, you can technically build a package using Python 3.9 which can also work on Python 3.8. The quickest and easiest way to ensure version compatibility would be to build it on the minimum version that you intend to support, so I would suggest building once using 3.8.
If you want to test that the package will work on different Python versions, you can use tox to run your automated tests on multiple versions in an easy way (automated tests are tests that you execute through pytest or unittest).
Here is how Python does its versioning and what the versions signify. Assuming that you are building the package to have it work on version 3.8 and all subsequent 3.x versions, then if it builds and works for 3.8, it will almost certainly work for 3.9, 3.10, 3.11 etc...
Although major versions such as 2.x and 3.x (and 4.x if it ever gets released) will have enough incompatible changes between them that it is not safe to assume that they are compatible.