What is being signed by the custom signer?

Question

I have the following code to sign, the problem I am having is that the data that will be signed is being send over the network and I need to process it on another function but im not sure what type of encoding or hash does it have?

CMSSignedDataGenerator signGenerator = new CMSSignedDataGenerator();
                X509Certificate userCert = (X509Certificate) this.certificateChain[0];
                ContentSigner mySigner = new CustomSigner(invoke,String.valueOf(userCert.getSerialNumber()),sad);
                signGenerator.addSignerInfoGenerator(
                        new JcaSignerInfoGeneratorBuilder(new JcaDigestCalculatorProviderBuilder().build())
                                .build(mySigner, userCert));
                signGenerator.addCertificates(new JcaCertStore(Arrays.asList(certificateChain)));
                CMSProcessableInputStream msg = new CMSProcessableInputStream(content);
                CMSSignedData signedData = signGenerator.generate(msg, false);
                return signedData.getEncoded();

What does bouncy castle send to sign and in what encoding is it? I recieve on the other side the following

MYGYMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTIxMDMyMzE3MTA0NVowLQYJKoZIhvcNAQk0MSAwHjANBglghkgBZQMEAgEFAKENBgkqhkiG9w0BAQsFADAvBgkqhkiG9w0BCQQxIgQgpJ/2Sz3j0sp6iqVKmyednqFjZ87SYEYhScT0sSKtHPU=

Answer 1

As Jim Garrison and dave_thompson_085 already said in comments that's a base64 encoded signed attributes DER structure with content type, signing time, message digest, and algorithm protection properties. Here an ASN.1 dump of it:

SET (4 elem)
  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.9.3 contentType (PKCS #9)
    SET (1 elem)
      OBJECT IDENTIFIER 1.2.840.113549.1.7.1 data (PKCS #7)
  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.9.5 signingTime (PKCS #9)
    SET (1 elem)
      UTCTime 2021-03-23 17:10:45 UTC
  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.9.52 cmsAlgorithmProtection (RFC 6211)
    SET (1 elem)
      SEQUENCE (2 elem)
        SEQUENCE (2 elem)
          OBJECT IDENTIFIER 2.16.840.1.101.3.4.2.1 sha-256 (NIST Algorithm)
          NULL
        [1] (2 elem)
          OBJECT IDENTIFIER 1.2.840.113549.1.1.11 sha256WithRSAEncryption (PKCS #1)
          NULL
  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.9.4 messageDigest (PKCS #9)
    SET (1 elem)
      OCTET STRING (32 byte) A49FF64B3DE3D2CA7A8AA54A9B279D9EA16367CED260462149C4F4B122AD1CF5

According to the algorithm protection attribute BouncyCastle expects that you return a SHA256withRSA signature value signing this structure.

In particular, as you wonder what type of encoding or hash does it have, the hash of this structure still will have to be calculated as part of your signing service.