After watching a YouTube video on the Diffie-Hellman Key Exchange, I wanted to try an implementation in JavaScript (Atwood's law).
I sketched up an cipher on Node.js with the following rules:
Step 1: Client and server agree on a shared key:
Client & server start with a 512bit prime public key pK
Client generates a 512bit prime private key kC and sends powMod(3, kC, pK)
Server generates a 512bit prime private key kS and sends powMod(3, kS, pK)
Client & Server use powMod(response, privatekey, pK) as the shared key
Step 2: Communication
Before a client sends data it is encrypted with the shared key using the Stanford Javascript Crypto Library (256bit AES, HMAC authentication, PBKDF2 password strengthening, and CCM authenticated-encryption.)
Once the server decrypts the data with the shared key, it generates a new 512bit prime private key and sends it as a SJCL encrypted response.
The client and server switch to a new shared key using powMod(3, prevSharedKey, newPrivKey)
Now I have a few questions..
How secure would such a system be in comparison with HTTPS or other algorithms? What are the weakest points of such a system?
In terms of security / practicality, would it be better to use 1024 bit keys for stronger security? Are the HMAC/PBKDF2/CCM options overkill? Is it worth modulating the shared key? Thanks for reading!
I've seen questions like this before – this is completely insecure for a number of reasons, foremost of which is the fact that it is impossible for a JavaScript client to verify that the server's key is authentic.
In a nutshell, without SSL, you are vulnerable to man-in-the-middle attacks. No browser-based JavaScript crypto implementation can overcome this fact.