HMAC-SHA256 in Python is producing different output to Nodejs

Question

I have a python script which uses the HMAC-hashlib.sha256 algorithm to generate a signature to authenticate over API.

# ZenEmu
import base64
import hashlib
import hmac
import json
from datetime import datetime
import requests

data = json.dumps(
    {
        "version": "1",
        "id": "284512",
        "created": "2023-01-17T02:26:29Z",
        "updated": "2023-01-17T02:26:29Z",
        "url": "https://hitchhikerio.zendesk.com/agent/tickets/284512",
        "type": "Question",
        "status": "Closed",
        "priority": "Low",
        "title": "[TEST4] Please Ignore!",
        "description": "first comment",
        "tags": "test",
        "requester_email": "[email protected]",
        "assignee_email": "[email protected]",
        "latest_comment": {
                "id": "6586310212693",
                "is_public": "true",
                "author_email": "[email protected]",
                "body": "second reply!",
        },

    }

).encode('utf-8')


sig = base64.b64encode(
    hmac.new(
        "secret".encode("utf-8"),
        "2022-10-10T04:42:00Z".encode("utf-8") + data,
        hashlib.sha256,
    ).digest()
)


print(sig)

This python file pirnts this result as the signature (sig) = b'DJXG/w1ARbM2ZOQ3pamMEkNY7qSJXpjbQMsARyWCr0Y='

I have tried converting this to Javascript with the help of my colleagues and referring to similar questions and resources online to no avail

This is my Javascript file

const crypto = require('crypto');
const utf8 = require('utf8')

const data = utf8.encode(JSON.stringify({
        version: "1",
        id: "284512",
        created: "2023-01-17T02:26:29Z",
        updated: "2023-01-17T02:26:29Z",
        url: "https://hitchhikerio.zendesk.com/agent/tickets/284512",
        type: "Question",
        status: "Closed",
        priority: "Low",
        title: "[TEST4] Please Ignore!",
        description: "first comment",
        tags: "test",
        requester_email: "[email protected]",
        assignee_email: "[email protected]",
        latest_comment: {
            id: "6586310212693",
            is_public: "true",
            author_email: "[email protected]",
            body: "second reply!",
        },
    }));

    console.log(data);

    const sig = crypto
        .createHmac('sha256', utf8.encode('secret'))
        .update(utf8.encode("2022-10-10T04:42:00Z") + data)
        .digest('base64')

    console.log(sig);

which prints the output Yqi7WvMPxdSlo5Vb9YlcbX2zu5aQpungXhuCYx3bc+4=

How do I get the Javascript code to generate the same output as the python script? (b'DJXG/w1ARbM2ZOQ3pamMEkNY7qSJXpjbQMsARyWCr0Y=')

EDIT: THE PYTHON FILE SHOULD STAY UNCHANGED, IT PRODUCES THE CORRECT OUTPUT.

Answer 1

Your JSON strings aren't identical. Python outputs the following:

'{"version": "1", "id": "284512", "created": "2023-01-17T02:26:29Z", "updated": "2023-01-17T02:26:29Z", "url": "https://hitchhikerio.zendesk.com/agent/tickets/284512", "type": "Question", "status": "Closed", "priority": "Low", "title": "[TEST4] Please Ignore!", "description": "first comment", "tags": "test", "requester_email": "[email protected]", "assignee_email": "[email protected]", "latest_comment": {"id": "6586310212693", "is_public": "true", "author_email": "[email protected]", "body": "second reply!"}}'

Node outputs the following:

'{"version":"1","id":"284512","created":"2023-01-17T02:26:29Z","updated":"2023-01-17T02:26:29Z","url":"https://hitchhikerio.zendesk.com/agent/tickets/284512","type":"Question","status":"Closed","priority":"Low","title":"[TEST4] Please Ignore!","description":"first comment","tags":"test","requester_email":"[email protected]","assignee_email":"[email protected]","latest_comment":{"id":"6586310212693","is_public":"true","author_email":"[email protected]","body":"second reply!"}}'

Notice how the Node version has no whitespace.

EDIT:

To fix it, you can use the Python function as follows:

json.dumps(your_data_object_here, separators=(",",":"))

Edit again:

To insert spaces into the Node version you can try the following:

JSON.stringify(obj).replace(/("[^"]+"[:,])/g, "$1 ");

That should insert spaces after all the : and ,, like in Python. Note that it does not handle escaped quote marks inside the strings. You'll need to update the Regex to deal with that.

Final edit:

Here's an alternate JS approach:

const data = JSON.stringify(obj, null, 1).match(/(\S.*)/gm).join("");

This constructs a "pretty-printed" JSON string with newlines and everything, then grabs each line starting at the first non-whitespace and joins them all up. That strips newlines and indentation, but preserves all whitespace between separators and inside strings. Should be sufficient.