From the doc , it said
To test in-app billing in an application you must install the
application on an Android-powered device. You cannot use the
Android emulator to test in-app billing.
But anyone have success in testing with Android x86 VM (http://www.android-x86.org)?
I suppose the VM should act like a real device and should be okay to test?
The actual platform doesn't really matter. IAB uses the Android Market/Play store application to provide its services, so if you have that installed and properly working, you will be able to test. If not, you won't. Since the IAB code is platform-independent, you don't really need to test it on all platforms you need to support.
Update 2022; However, OP asks about android-x86 project, because the Android-Emulator's standard system-images (those provided by Google) do somehow notify Google-Wallet and/or Google-Pay app, that they are under Emulator, which causes said apps to NOT even allow adding payment-method.
See How to test In-app Billing on an emulator?
This behavior is required, because else any restricted country could switch to Emulator.