Can I upload a custom app in my teams for tenant "A"(I have the upload custom app link enabled by teams admin) which is not published to the Teams App store yet.
The custom app contains a bot registered with Microsoft bot framework and does not contain tabs.
The Microsoft App Registration for this app is registered in other tenant "B" and the app doesn't require any special delegated or application permissions to Microsoft graph
Yes this should work no problem. Depending on what you app includes (Bot, Tab, etc.) these should work fine, provided:
the user's device can access the system no problem (e.g. if your tab is hosted on a public httpS url, versus let's say an internal-only address that only people in tenant "A" could access
you don't have any tenant-specific filtering on (for instance, there are bot samples that show how to -restrict- your bot to only users from a specific tenant). This is NOT the default though, you would have put this into your bot specifically. By default it will work fine across tenants.