Xero Webhook verification failing on NodeJS/Express backend

Question

I am attempting to setup a webhook to the demo Xero Account in a NodeJS and Express environment. Seemingly the Webhooks key Xero provides to use is not matching the Intent to receive tests that they send to my server.

I have followed the guide provided by the Xero team on their Medium article from three years ago. This was retrieved from the Xero-Node Hooks section I have copied the code without using classes to get this result:

import * as crypto from "crypto";
import express from "express";
import { Request, Response } from "express";

const app = express();

app.use("/webhooks", express.raw({ type: "application/json" }));
app.use(express.json());

function verifyWebhookEventSignature(req: Request) {
  const computedSignature = crypto
    .createHmac(
      "sha256",
      process.env.XERO_WEBHOOK_KEY
    )
    .update(req.body.toString())
    .digest("base64");
  const xeroSignature = req.headers["x-xero-signature"];

  if (xeroSignature === computedSignature) {
    console.log("Signature passed! This is from Xero!");
    return true;
  } else {
    console.log(
      "Signature failed. Webhook might not be from Xero or you have misconfigured something..."
    );
    console.log(
      `Got {${computedSignature}} when we were expecting {${xeroSignature}}`
    );
    return false;
  }
}

app.post("/webhooks", async (req: Request, res: Response) => {
  console.log(
    "webhook event received!",
    req.headers,
    req.body,
    JSON.parse(req.body)
  );
  return verifyWebhookEventSignature(req)
    ? res.status(200).send()
    : res.status(401).send();
});

app.listen(5000, () => {
  console.log(`Server is running on port ${5000}`);
});

On the above example, the endpoint is reached after clicking the "Intent to receive" button on the Xero Webhook page. But I never succeed past the Response to incorrectly signed payload not 401. Last sent at 2023-11-27 18:00:53 UTC. A couple of times I have gotten the "Signature passed! This is from Xero!" But the outcome still seems to be the same where the end result is not what Xero expected and spits out that above error again with Response to incorrectly signed payload not 401. Last sent at 2023-11-27 18:00:53 UTC.

ngrok reports: POST /webhooks 401 Unauthorized POST /webhooks 401 Unauthorized

I have also attempted an older guide from them again on the same blog section of Xero but unfortunately had the same result.

Doing a deeper dive the signature and computed signature seem to be relatively similar, but not the exact same. Here was a response from one of the failed examples (Webhook key has been changed after posting): {4i4UlslXwPfrYZNMoV1IIL52UdW8M3QEZMadUgvJ2Yc=} when we were expecting {4i4UlslXwPfrYZNMo/1IIL52UdW8M3QEZMadUgvJ2Yc=}

A form post mentioned an issue with slashes on n8n where their suggestion was to replace the slashes. I attempted this both with global string replacement and single string replacement. This did remove the slashes but as evident in the above example it appears that the actual characters are different where the slash is from the computed signature and xero signature.

I have also attempted to regenerate the key by deleting the webhook from Xero and recreating it with the Ngrok URL a few times but am getting the same result.

Here is an example I received with both the signature pass and signature fail, but still receiving the ICR from xero:

Server is running on port 5000
webhook event received! {
  host: 'eeb6-81-106-82-109.ngrok.io',
  'content-length': '95',
  'content-type': 'application/json; charset=utf-8',
  'x-forwarded-for': '52.20.64.83',
  'x-forwarded-host': 'eeb6-81-106-82-109.ngrok.io',
  'x-forwarded-proto': 'https',
  'x-xero-signature': 'kQtanVOgzl9tEdBTrgDDAv9Thr+ThIGdDDDVFU5L/ZU=',
  'accept-encoding': 'gzip'
} <Buffer 7b 22 65 76 65 6e 74 73 22 3a 5b 5d 2c 22 66 69 72 73 74 45 76 65 6e 74 53 65 71 75 65 6e 63 65 22 3a 20 30 2c 22 6c 61 73 74 45 76 65 6e 74 53 65 71 ... 45 more bytes> {
  events: [],
  firstEventSequence: 0,
  lastEventSequence: 0,
  entropy: 'EDEKUDLFUPIMATQBJWMR'
}
Signature passed! This is from Xero!
webhook event received! {
  host: 'eeb6-81-106-82-109.ngrok.io',
  'content-length': '95',
  'content-type': 'application/json; charset=utf-8',
  'x-forwarded-for': '52.20.64.83',
  'x-forwarded-host': 'eeb6-81-106-82-109.ngrok.io',
  'x-forwarded-proto': 'https',
  'x-xero-signature': 'kQtanVOgzl9tEdBTrgDDAv9Thr+ThIGdDDDV/U5L/ZU=',
  'accept-encoding': 'gzip'
} <Buffer 7b 22 65 76 65 6e 74 73 22 3a 5b 5d 2c 22 66 69 72 73 74 45 76 65 6e 74 53 65 71 75 65 6e 63 65 22 3a 20 30 2c 22 6c 61 73 74 45 76 65 6e 74 53 65 71 ... 45 more bytes> {
  events: [],
  firstEventSequence: 0,
  lastEventSequence: 0,
  entropy: 'EDEKUDLFUPIMATQBJWMR'
}
Signature failed. Webhook might not be from Xero or you have misconfigured something...
Got {kQtanVOgzl9tEdBTrgDDAv9Thr+ThIGdDDDVFU5L/ZU=} when we were expecting {kQtanVOgzl9tEdBTrgDDAv9Thr+ThIGdDDDV/U5L/ZU=}

Any help/advice would be massively appreciated. Many thanks

Answer 1

UPDATE:

After many hours of further testing the issue appeared to be ngrok for me. For reference the ngrok command I was running was just ngrok http 5000. No errors were displayed during setup and testing with firewall on Windows (22H2) enabled/disabled still returned the same results.

I unfortunately struggled to find an alternative to ngrok and gave multiple others a try from this list on github https://github.com/anderspitman/awesome-tunneling but in the end the one that worked first try for me was Loophole by doing loophole http 5000 (required an account to setup seemingly).

With ngrok

With Loophole

Xero mentions that for the webhooks to succeed certain criteria must be met. My assumptions are that there could have been some subtle differences between how various tunnelling solutions handle webhook requests and this caused issues when validating the ICR.