Search code examples
javacryptographybouncycastlesignatureecdsa

JAVA How to verify ECDSA signature when r or s are negative


I'm writing a chunk of code to check the signature

My code below works if r and s are positive if not the verify fails.

My code is

PublicKey publicKey = cert.getPublicKey();
Signature ecdsaVerify = Signature.getInstance("SHA256withECDSA", new BouncyCastleProvider());
ecdsaVerify.initVerify(publicKey);
ecdsaVerify.update(message.getBytes());
valid = ecdsaVerify.verify(sigDer);

My signature is created, starting from raw r and s, in der format keep in mind if r or s are negative to add '00' to der format, I've also used the BouncyCastle ANS1 class

ASN1EncodableVector v = new ASN1EncodableVector();
v.add(new ASN1Integer(r));
v.add(new ASN1Integer(s));          
DERSequence der = new DERSequence(v);

but still I'm able to verify if r and s are positive i cannot if one of them is negative (the initial byte is greater than 0x80)

what i've missing?

-----EDIT-----

I've created a better code to explain my problem

static boolean verifySignature(String messagePlain, String certificatePEM, String rHex, String sHex)
{
    boolean valid = false;
    Security.addProvider(new BouncyCastleProvider());
        
    try 
    {       
        BigInteger r = new BigInteger(rHex, 16);
        BigInteger s = new BigInteger(sHex, 16);

        ASN1EncodableVector v = new ASN1EncodableVector();
        v.add(new ASN1Integer(r));
        v.add(new ASN1Integer(s));          
        DERSequence der = new DERSequence(v);
        byte[] signature = der.getEncoded();

        CertificateFactory cf = CertificateFactory.getInstance("X.509", new BouncyCastleProvider());
        InputStream certStream = new ByteArrayInputStream(certificatePEM.getBytes());
        X509Certificate x509cert  = (X509Certificate)cf.generateCertificate(certStream);
            
        PublicKey publicKey = x509cert.getPublicKey();
        Signature ecdsaVerify = Signature.getInstance("SHA256withECDSA", new BouncyCastleProvider());
        ecdsaVerify.initVerify(publicKey);
        ecdsaVerify.update(messagePlain.getBytes());
        valid = ecdsaVerify.verify(signature);

    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (CertificateException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (SignatureException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (NoSuchAlgorithmException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    } catch (InvalidKeyException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    return valid;
}

My Main

public static void main( String[] args )
{       
    String messNOT = "dc03d9c5676cc8a76b59214b86214b865d01022cdd518622c721967c7c23abbc133c133c133c133c133c133c2c651a2419a64d4a598d19cd202326752cdefe3204031f00000506675b20383373";
    String rNOT = "663e5dd196f22f03e3aba8f6652d1ca92d207088911987f44048a4b85ec36e91";
    String sNOT = "84c4ca1678dce85b19b2ddb378f4a1df878cec21ce6fbab9292576281602041d";
    String certNOT ="-----BEGIN CERTIFICATE-----\r\n"
            + "MIICzDCCAlOgAwIBAgICANMwCgYIKoZIzj0EAwMwdjELMAkGA1UEBhMCQ0gxDjAM\r\n"
            + "BgNVBAoMBUFkbWluMREwDwYDVQQLDAhTZXJ2aWNlczEiMCAGA1UECwwZQ2VydGlm\r\n"
            + "aWNhdGlvbiBBdXRob3JpdGllczEgMB4GA1UEAwwXYWJuYS1jc2NhLXN3aXR6ZXJs\r\n"
            + "YW5kLTIwHhcNMjIwMjA5MTQxNDM2WhcNMjgwMjE0MTQxNDM2WjAaMQswCQYDVQQG\r\n"
            + "EwJDSDELMAkGA1UEAxMCVlMwggEzMIHsBgcqhkjOPQIBMIHgAgEBMCwGByqGSM49\r\n"
            + "AQECIQCp+1fboe6pvD5mCpCdg41ybjv2I9UmICggE0gdH25TdzBEBCB9Wgl1/Cww\r\n"
            + "V+72dTBBev/n+4BVwSbcXGzpSktE8zC12QQgJtxcbOlKS0TzMLXZu9d8v5WEFilc\r\n"
            + "9+HOa8zcGP+MB7YEQQSL0q65y35XyyxLSC/8gbevud4n4eO9I8I6RFO9ms4yYlR+\r\n"
            + "+DXD2sT9l/hGGhRhHcnCd0UTLe2OVFwdVMcvBGmXAiEAqftX26Huqbw+ZgqQnYON\r\n"
            + "cYw5eqO1Yab3kB4OgpdIVqcCAQEDQgAEEw6HdjvJQhXyiitwaWZ4DmypigCjdJen\r\n"
            + "oRjW7Tz3n1Mlqf2dwwgYcwHB7N6zvLpQ2aZmILOicbyvSKHCZHaztqNRME8wHwYD\r\n"
            + "VR0jBBgwFoAU5/zrAS4O3jj5DvGKXM5XDxMEyXMwFQYHZ4EIAQEGAgQKMAgCAQAx\r\n"
            + "AxMBVjAVBgNVHSUBAf8ECzAJBgdngQgBAQsBMAoGCCqGSM49BAMDA2cAMGQCMEG4\r\n"
            + "zJn2/N85NLsDM58+jB0oyFTB152gPyAcdhq04gLpXynW2qbSNfE6Fgw34Qm+vAIw\r\n"
            + "AZSAdepOu4r6T+hj1p8Q3HFlSvlByjpr/b2VNiMXup6v3O7BnGqbyRFgMlc3c/BB\r\n"
            + "-----END CERTIFICATE-----";
        
    String messYES = "dc03d9c5d9b71fe6fe336de4e53214665d01022cdb5bd2b3c549cd1da93c5bd458135c6f57fc133c133c133c9e2e4d0d28043132b09e1ae53f5c52853f78fe370403900000050659e9269f2cb8";
    String rYES = "473c2e821b49bffc32e49cb7ed5f0645a32ad76c8258ebd4ec9aca56";
    String sYES = "77c7e16920143444cf68d3891436b1a0189e08d06b104fdb247cd3ef";
    String certYES ="-----BEGIN CERTIFICATE-----\r\n"
            + "MIID7jCCAdagAwIBAgIBAjANBgkqhkiG9w0BAQsFADAuMQswCQYDVQQGEwJVVDEP\r\n"
            + "MA0GA1UECgwGVVRPUElBMQ4wDAYDVQQDDAVDU0NBMTAeFw0yMTEwMjAxMjEwMTha\r\n"
            + "Fw0yNDEwMTkxMjEwMThaMBoxCzAJBgNVBAYTAlVUMQswCQYDVQQDDAJBMTCCARMw\r\n"
            + "gdQGByqGSM49AgEwgcgCAQEwKAYHKoZIzj0BAQIdANfBNKomQ2aGKhgwJXXR14ew\r\n"
            + "nwdXl9qJ9X7IwP8wPAQcaKXmLKnObBwpmAOmwVMLUU4YKtiwBCpZytKfQwQcJYD2\r\n"
            + "PM/kQTiHBxOxqSNp4z4hNdJm27NyOGxACwQ5BA2QKa0sflz0NAgjsqh9xoyeTOMX\r\n"
            + "TB5u/e4SwH1Yqlb3csBybyTGuJ5OzawkNUuemcqj9tN2FALNAh0A18E0qiZDZoYq\r\n"
            + "GDAlddD7mNEWvEtt3ryjpaeTnwIBAQM6AATBWidMSa55454DGh7j+IkWfswMr33v\r\n"
            + "jwQiNONEo+LRC3+yL5dDcfdCfDQqrMdAC8PfR/8cMVRMJ6M6MDgwHwYDVR0jBBgw\r\n"
            + "FoAUCoYRgjvodzM4CDLB5bSdQMEXyBIwFQYDVR0lAQH/BAswCQYHZ4EIAQEOAjAN\r\n"
            + "BgkqhkiG9w0BAQsFAAOCAgEAe+CFC4CEPjGx9vDi+zWKCho6QtwGI/MVwWqmgoRS\r\n"
            + "UgBgMrtAq/tYeWz8tD6kZzLW8M5PxGLRUyd4oBt2WzdNqHGJ72w4NDeUm33DH40g\r\n"
            + "bivStdcHIeYxxNCq1WFiN1oAfqpi+XdzfSgRSSuYY+8sZoGxg2A7ydMIGN4CuWGD\r\n"
            + "Op1T/BViespPkZWFo9P8wdetaV3mqjLu7oiDISkfxQhGRFMpOJrw/blxhl5ZgRSh\r\n"
            + "Zrvpz8nzMZitxPompJLCL0rHCofLwPRKvFF4W5DQt0n3MWVrXjHtLFC4haO05Ugw\r\n"
            + "VQy+KgWwE+AhdkUJGHrSUtEajiscgULU6l/UIuOpmF68VqHvx6EEM7UpLbXXQkuM\r\n"
            + "lE+9YWKRj0bssVKtoShw3m+TDsrk1xfGzk6lLPPpcObvdNkRiPufoMCD0Ci8h05S\r\n"
            + "oQCWLFLJcOrn8yGVSwaHgbcsuVX15rCneeXIhGbRlnnqjD/Z+vWGh3RleOSvdIDM\r\n"
            + "hnYDoRm7gZs0b5hIZ6M9kBqDTf61rraV7+0LzFuWMIq7+ree2eCIeBv26f+LRFXy\r\n"
            + "UDGWqEfiZR2ytyY54VUc/ptMVwkz1EethHu2JqZxAp1qkWUci++AZN43Typ/NJ9r\r\n"
            + "qZjM5nSSQzQDVvfE8seLe/78mmwmM9pD+NMGxWMpdA0MBwJN0Nzhlcp/rnWKtp7u\r\n"
            + "3jo=\r\n"
            + "-----END CERTIFICATE-----";
                    
    System.out.println("valid: " + verifySignature(messNOT, certNOT, rNOT, sNOT));
    System.out.println("valid: " + verifySignature(messYES, certYES, rYES, sYES));
}

The output

valid: false
valid: true

using openssl can verify both signature why it does not work in java ?


Solution

  • The reason of the issue is not that r and/or s start with a byte >= 0x80, but that the message messNOT is UTF-8 encoded during verification.

    The verification for the problematic message messNOT is successful if messNOT is not UTF-8 encoded but hex decoded. In contrast, for the working message messYES, verification is successful if messYES is UTF-8 encoded.

    Since both messages are UTF-8 encoded in the posted code, this explains why the verification of messNOT fails while that of messYES succeeds.


    In order for verification to succeed for both messages, the simplest way is to pass the message as byte[] in verifySignature():

    static boolean verifySignature(byte[] messagePlain, String certificatePEM, String rHex, String sHex) {
        ...
        ecdsaVerify.update(messagePlain);
        ...
    }
    

    and perform the conversion beforehand (as described):

    import org.bouncycastle.util.encoders.Hex;
    ...
    System.out.println("valid: " + verifySignature(Hex.decode(messNOT), certNOT, rNOT, sNOT)); // valid: true
    System.out.println("valid: " + verifySignature(messYES.getBytes(StandardCharsets.UTF_8), certYES, rYES, sYES)); // valid: true
    

    This is consistent with verification via OpenSSL, as it should be.


    By the way, BouncyCastle also supports the IEEE P1363 (i.e. r|s) format with SHA256withPlain-ECDSA, so the conversion to ASN.1/DER is not necessary.