I'm trying to use the public key encryption in crypto.js, I want to encrypt some message using publicEncrypt, and decrypt it with privateDecrypt.
const crypto=require('crypto');
let alice=crypto.getDiffieHellman('modp14');
alice.generateKeys();
let enc=crypto.publicEncrypt(alice.getPublicKey(),Buffer.from('hello'));
However, the crypto.publicEncrypt line is causing the following error:
"Error: error:0906D06C:PEM routines:PEM_read_bio:no start line"
The public key value returned by crypto.DiffieHellman.getPublicKey() is just the raw DH number, optionally encoded in base64 or hex. It is not in (any) PEM format or even ASN.1/DER format (which could easily be turned into PEM). Similarly crypto.ECDH.getPublicKey() is only the point (in conventional X9.62 format), not any PEM or DER format.
Moreover, DH and ECDH are not encryption algorithms, they are key-agreement (or secret-agreement) algorithms, and that operation is performed by DiffieHellman.computeSecret() or ECDH.computeSecret() respectively. Although not clearly documented, publicEncrypt actually calls OpenSSL's EVP_PKEY_encrypt{_init,} which doesn't support DH or ECDH, only RSA (with several choices of padding) and possibly GOST-wrap (I can't easily verify that and it may well be version dependent because upstream OpenSSL as of 1.1.0 a few years ago dropped the GOST algorithms).
In short, you can't do that.