"crypto/rsa: decryption error" when trying to use rsa encryption and decryption in dart (pointycastle) and golang

Question

I'm trying to encrypt a message in dart (using pointycastle) and decrypt it in golang (using the standard library). The private and public keys do match. The message is sent over TCP.

dart code:

// import 'package:encrypt/encrypt.dart' as enc;
final publicKey =
    enc.RSAKeyParser().parse(serverRsaPublicKey) as RSAPublicKey;

final rsaEncrypter = AsymmetricBlockCipher('RSA/OAEP')
  ..init(true, PublicKeyParameter<RSAPublicKey>(publicKey));
final ciphertext =
    rsaProcessInBlocks(rsaEncrypter, Uint8List.fromList(utf8.encode('Some message')));

tcpSendMessage(ciphertext); // Dummy function

rsaProcessInBlocks is the function used in the pointycastle's rsa tutorial (https://github.com/bcgit/pc-dart/blob/master/tutorials/rsa.md - _processInBlocks)

golang code:

/*
import (
    "crypto/rand"
    "crypto/rsa"
    "crypto/sha256"
    "crypto/x509"
    "encoding/pem"
)
*/

var block *pem.Block
block, _ = pem.Decode([]byte(RSA_PRIVATE_KEY))
privateKey, err := x509.ParsePKCS1PrivateKey(block.Bytes)
if err != nil {
    println(err.Error())
    return
}

ciphertext := TcpGetMessage() // Dummy function
plaintext, err := rsa.DecryptOAEP(sha256.New(), rand.Reader, privateKey, ciphertext, []byte(""))
if err != nil {
    println(err.Error()) // Error happens here
    return
}

I'm getting the following error in golang: crypto/rsa: decryption error.

I tested the encryption and decryption independently in each language and it works alright. I also tested whether the message is sent properly over tcp (which it is)

I'm guessing that a different algorithm is used internally or that the libraries are using different PKCS versions

I tried looking deeper into the error but golang doesn't reveal it for security reasons.

Any help would be greatly appreciated.

Answer 1

as Topaco (https://stackoverflow.com/users/9014097/topaco) said, the digest in the dart code is not explicitly specified, so I have to use OAEPEncoding.withSHA256(RSAEngine()) instead of AsymmetricBlockCipher('RSA/OAEP')