I have a Golang program working with Twirp and I want to create GraphQl server, but as far as I know, twirp build on top of the Rest API
You Just have to make a wrapper over the Twirp based RPC API.
This is a similar case to wrapping a GraphQL API over a Rest API.
You should also read this article, where the wrapping of a graphql api over a rest one is shown.