Desired behaviour of my app would be: user that has my app installed can invite other users to install it with a customized request "One of your friends invites to install MyApp". And the requester would be my app (like with notifications API), so kind of anonymously.
Is it possible to customize Request Dialog to do this, or use any other technology?
App generated requests can only be sent to users that have authenticated your application. This is a security measure. If any application could send requests to any user the possibilities of spam would be astronomical.
Furthermore, a user would want to know who invited him to use the application no? What you are suggesting would be extremely problematic.
So to answer your question, no. This is not possible. Application requests to non-authorized users must be sent by authorized users.