EOT,ETX in base64 encoded bouncy castle signature

Question

I have a function that uses Bouncy Castle libraries (version bouncy castle: jdk15on-1.57 and java 8) to sign a string and return the base64 value of the signed string (including the content of the string).

Part of the code:

        String stringMessage = "exampleContentBlaBla";

            // Load the keystore
            KeyStore keystore = KeyStore.getInstance("JKS");
           
            keystore.load(new FileInputStream("path/to/keystore.jks"), "keystorePassword".toCharArray());

            // Get the private key and certificate
            String alias = "yourAlias";
            PrivateKey privateKey = (PrivateKey) keystore.getKey(alias, "keyPassword".toCharArray());
            Certificate certificate = keystore.getCertificate(alias);

            // Initialize CMS components
            CMSSignedDataGenerator generator = new CMSSignedDataGenerator();
            CMSTypedData content = new CMSProcessableByteArray(stringMessage.getBytes("UTF-8"));

            // Create a JcaSimpleSignerInfoGeneratorBuilder and configure the signer
            JcaContentSignerBuilder contentSignerBuilder = new JcaContentSignerBuilder("SHA256withRSA");
            ContentSigner contentSigner = contentSignerBuilder.build(privateKey);

            JcaSignerInfoGeneratorBuilder builder = new JcaSignerInfoGeneratorBuilder(
                    new JcaDigestCalculatorProviderBuilder().setProvider("BC").build());
            generator.addSignerInfoGenerator(builder.build(contentSigner, certificate));

            // Generate the signed data
            CMSSignedData signedData = generator.generate(content, true);

            // Encode the signed data to Base64
            byte[] pkcs7Bytes = signedData.getEncoded();
            String encodedData = new String(Base64.encode(pkcs7Bytes), "UTF-8");

However, when I decode the result, the string contains random characters like EOT, ETX, and x82, making it impossible to interpret the content of the signed string, even though the signature verification passes successfully.

Example:

outcome of base64 decoded signature

    String testDecoded = new String(Base64.decode(encodedString.getBytes("UTF-8")));
                    System.out.println(testDecoded );

This way I see the decoded message in the console. And I also used notepad++ tool for decoding for better visualization

I have pinpointed that the EOT and ETX characters are randomly added after the signature. I tried base64 encoding and decoding the string without the signature, and there were no issues.

I have explicitly added UTF-8 encoding wherever possible. Something happens during the signing process, but I can't find a logical explanation or how to fix it.

Can anyone help me understand what is happening and how to resolve this issue?

Answer 1

I've been debugging the Bouncy Castle classes and pinpointed where the issue lies.

The EOT and ETX characters were added after this line:

// Encode the signed data to Base64
byte[] pkcs7Bytes = signedData.getEncoded();

The fix was to explicitly define the encoding as "DL" or "DER".

 byte[] pkcs7Bytes = signedData.getEncoded("DL");

Possible encodings you can use here are "BER", "DL", or "DER".

By default, "DER" should be used, but if it's not specified, EOT and ETX characters are added.

Site that helped me during the research: A Layman's Guide to a Subset of ASN.1, BER, and DER

Note that I have upgraded the BC version to 18on-1.78.1, so I am not sure if the mentioned fix would work for lower versions, that should be tested additionally.