CipherInputStream gets BadPaddingException

Question

Hope someone can see my issue. I'm trying to send encrypted data over the network using CipherOutputStream/CipherInputStream and keep on getting BadPaddingException. I use the same RSA keys for both sides and can manually encrypt/decrypt fine. The modulus for both private and public keys match.

This is the working example that manually encrypt/decrypt using Cipher and the keys.

public static void main(String[] args) {
    PrintStream out = System.out;
    try {
        PrivateKey pri_key = Util.readPrivateKey("data/private.der");
        PublicKey pub_key = Util.readPublicKey("data/public.der");

        Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");

        String hi = "Hello World";
        byte[] data = hi.getBytes("UTF8");
        out.println("data[" + data.length + "]");

        cipher.init(Cipher.ENCRYPT_MODE, pub_key);
        byte[] encdata = cipher.doFinal(data);
        out.println("encdata[" + encdata.length + "]");

        cipher.init(Cipher.DECRYPT_MODE, pri_key);
        byte[] decdata = cipher.doFinal(encdata);
        out.println("decdata[" + decdata.length + "]");
        out.println(new String(decdata, "UTF8"));
    } catch (Exception e) {
        e.printStackTrace();
    }
}

The send/receive codes that does the encrypt/decrypt

static class ByteEncComm extends Comm {
    public ByteEncComm(Socket sock, PrivateKey priv_key, PublicKey pub_key) {
        super(sock, priv_key, pub_key);
    }

    public void send(Bird b) {
        try {
            Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");
            cipher.init(Cipher.ENCRYPT_MODE, public_key);

            byte[] data = "Hello World".getBytes("UTF8");
            CipherOutputStream cos = new CipherOutputStream(sock.getOutputStream(), cipher);
            cos.write(data, 0, data.length);
            cos.flush();
            out.println("Sent data[" + data.length + "]");
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public Bird receive() {
        PrintStream out = System.out;
        try {
            Cipher cipher = Cipher.getInstance("RSA/ECB/OAEPWithSHA1AndMGF1Padding");
            cipher.init(Cipher.DECRYPT_MODE, private_key);

            CipherInputStream cis = new CipherInputStream(sock.getInputStream(), cipher);
            out.println("Avail " + cis.available());
            byte[] data = cis.readAllBytes();
            out.println("Got data[" + data.length + "]");
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

Client Code:

public class Client {
    public static void main(String[] args) {
        PrintStream out = System.out;
        try {
            PrivateKey pri_key = Util.readPrivateKey("data/private.der");
            PublicKey pub_key = Util.readPublicKey("data/public.der");

            Socket sock = new Socket("127.0.0.1", 5555);

            Comm comm = new Comm.ByteEncComm(sock, pri_key, pub_key);
            comm.send(new Bird("Robin"));

            sock.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Server code:

public class Server extends Thread {
    public static void main(String[] args) {
        PrintStream out = System.out;
        try {
            PrivateKey pri_key = Util.readPrivateKey("data/private.der");
            PublicKey pub_key = Util.readPublicKey("data/public.der");

            ServerSocket serversock = new ServerSocket(5555);
            Socket sock = serversock.accept();

            Comm comm = new Comm.ByteEncComm(sock, pri_key, pub_key);
            comm.receive();

            sock.close();
            serversock.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

Error:

java.io.IOException: javax.crypto.BadPaddingException: Decryption error
    at java.base/javax.crypto.CipherInputStream.getMoreData(CipherInputStream.java:148)
    at java.base/javax.crypto.CipherInputStream.read(CipherInputStream.java:261)
    at java.base/java.io.InputStream.readNBytes(InputStream.java:409)
    at java.base/java.io.InputStream.readAllBytes(InputStream.java:346)
    at foo.enc.Comm$ByteEncComm.receive(Comm.java:139)
    at foo.enc.Server.main(Server.java:21)
Caused by: javax.crypto.BadPaddingException: Decryption error
    at java.base/sun.security.rsa.RSAPadding.unpadOAEP(RSAPadding.java:488)
    at java.base/sun.security.rsa.RSAPadding.unpad(RSAPadding.java:284)
    at java.base/com.sun.crypto.provider.RSACipher.doFinal(RSACipher.java:372)
    at java.base/com.sun.crypto.provider.RSACipher.engineDoFinal(RSACipher.java:418)
    at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2152)
    at java.base/javax.crypto.CipherInputStream.getMoreData(CipherInputStream.java:145)
    ... 5 more

Answer 1

If you are using CipherOutputStream, you should take a look at the close() function, the function implicitly performs a doFinal() (Source Link):

Closes this output stream and releases any system resources associated with this stream.

This method invokes the doFinal method of the encapsulated cipher object, which causes any bytes buffered by the encapsulated cipher to be processed. The result is written out by calling the flush method of this output stream.

This method resets the encapsulated cipher object to its initial state and calls the close method of the underlying output stream.

---

It may be better for you to simply make use of the basic OutputStream and InputStream if CipherOutputStream is not mandatory. The extra effort is to perform your encryption with doFinal() manually before you send it out.