There is a server (fastAPI), it has endpoints: users/items/etc.
Is it true that different endpoints(users/items/etc.) are different microservice
This question does not have a general answer. It depends on your system deployment architecture.
In a straight forward setup you would publish microservices directly on a network, which usually means each service will be available under a different address/name or at least port. For example in a kubernetes setup each service is made available as kubernetes deployment/service and has a name in the internal dns.
But in more complex deployments it would not be unusual to deploy a gateway service that has to be used in order to reach the microservices. The gateway service may fulfill some functions, such as security, metrics and adding resilience. Since the gateway service may be reachable under a single address/name and port, it could use different HTTP paths for providing access to the microservices (like you describe in your question). So in reality the gateway service may forward the requests to separately deployed services. And note that the gateway itself may be available under a single address/name, but could be deployed as a distributed service itself (see for example how AWS API Gateway service would be implemented).
In order to learn more about such patterns, I recommend to research more about the idea of a "service mesh" and API Gateways.