How to make my API private but usable by mobile application?

Here is my requirements:

Usable by any mobile application I'm developing

I'm developing the mobile application, therefore I can implement any securing strategies.

Cacheable using classical HTTP Cache strategy

I'm using Varnish with a very basic configuration and it works well

Not publicly available

I don't want people be able to consume my API

Solutions I think of:

Use HTTPS, but it doesn't cover the last requirements because proxying request from the application will show the API KEY used.

Is there any possibility to do this? Using something like a private/public key for example?

Which fits well with HTTP, Apache, and Varnish.

Solution

There is no way to ensure that the other end of a network link is your application. This is not a solvable problem. You can obfuscate things with certificates, keys, secrets, whatever. But all of these can be reverse-engineered by the end user because they have access to the application. It's ok to use a little obfuscation like certificates or the like, but it cannot be made secure. Your server must assume that anyone connecting to it is hostile, and behave accordingly.

It is possible to authenticate users, since they can have accounts. So you can certainly ensure that only valid users may use your service. But you cannot ensure that they only use your application. If your current architecture requires that, you must redesign. It is not solvable, and most certainly not solvable on common mobile platforms.

If you can integrate a piece of secure hardware, such as a smartcard, then it is possible to improve security in that you can be more certain that the human at the other end is actually a customer, but even that does not guarantee that your application is the one connecting to the server, only that the smartcard is available to the application that is connecting.

For more on this subject, see Secure https encryption for iPhone app to webpage.