Implementing the PHP methods KSORT and HTTP_BUILD_QUERY in Python on Nested Dicts

I need to re-create the below code in Python to verify the signature included in the header of an API response.

  <?php
  // Profile Key (ServerKey)
  $serverKey = "SRJNLKHG6K-HWMDRGW66J-LRWTGDGRNK"; // Example

  // Request body include a signature post Form URL encoded field
  // 'signature' (hexadecimal encoding for hmac of sorted post form fields)
  $requestSignature = $_POST["signature"];
  unset($_POST["signature"]);
  $signature_fields = $_POST;
  
  // Ignore empty values fields
  $signature_fields = array_filter($signature_fields);
  
  // Sort form fields 
  ksort($signature_fields);

  // Generate URL-encoded query string of Post fields except signature field.
  $query = http_build_query($signature_fields);

  $signature = hash_hmac('sha256', $query, $serverKey);
  if (hash_equals($signature,$requestSignature) === TRUE) {
    // VALID Redirect
    // Do your logic
  }else{
    // INVALID Redirect
    // Log request
  }
?>

I have 0 knowledge of PHP and I've been trying for the last 24 hours to re-create it myself at no avail.

The part I'm stuck on is the ksort and the http_build_query methods.

What I'm trying to understand is what will the result of these methods look like for a NESTED array such as:

    {
  "tran_ref": "TST2014900000688",
  "cart_id": "Sample Payment",
  "cart_description": "Sample Payment",
  "cart_currency": "AED",
  "cart_amount": "1",
  "customer_details": {
    "name": "John Smith",
    "email": "[email protected]",
    "phone": "9711111111111",
    "street1": "404, 11th st, void",
    "city": "Dubai",
    "state": "DU",
    "country": "AE",
    "ip": "127.0.0.1"
  },
  "payment_result": {
    "response_status": "A",
    "response_code": "831000",
    "response_message": "Authorised",
    "acquirer_message": "ACCEPT",
    "acquirer_rrn": "014910159369",
    "transaction_time": "2020-05-28T14:35:38+04:00"
  },
  "payment_info": {
    "card_type": "Credit",
    "card_scheme": "Visa",
    "payment_description": "4111 11## #### 1111"
  }
}

And if there are readily available methods in python that can accomplish this that would be really helpful.

Thanks as always.

Solution

In response to your question about ksort() and http_build_query(), here are what the results look like when using PHP 7.3 and the following code (based on your given code/data given). First, please note that when I ran ksort() on your example data, it appears that ksort() sorts the initial keys in the data, but the inner keys don't seem to be sorted.

In order to sort the inner keys as well, I wrote a simple function called sort_again() that appears to sort the inner keys. The results of the plain ksort() versus the sort_again() results are below the following example PHP code.

<?php

$jsonData = '
{
  "tran_ref": "TST2014900000688",
  "cart_id": "Sample Payment",
  "cart_description": "Sample Payment",
  "cart_currency": "AED",
  "cart_amount": "1",
  "customer_details": {
    "name": "John Smith",
    "email": "[email protected]",
    "phone": "9711111111111",
    "street1": "404, 11th st, void",
    "city": "Dubai",
    "state": "DU",
    "country": "AE",
    "ip": "127.0.0.1"
  },
  "payment_result": {
    "response_status": "A",
    "response_code": "831000",
    "response_message": "Authorised",
    "acquirer_message": "ACCEPT",
    "acquirer_rrn": "014910159369",
    "transaction_time": "2020-05-28T14:35:38+04:00"
  },
  "payment_info": {
    "card_type": "Credit",
    "card_scheme": "Visa",
    "payment_description": "4111 11## #### 1111"
  }
}
';

$jsonArray = json_decode($jsonData, $flags=JSON_OBJECT_AS_ARRAY);

ksort($jsonArray);
echo "ksort() results:\n";
print_r($jsonArray);

// ref: q16.py
function sort_again($arr) {
    ksort($arr);
    $result = [];
    foreach ($arr as $key => $value) {
        if (is_array($value)) {
            $result[$key] = sort_again($value);
        } else {
            $result[$key] = $value;
        }
    }
    return $result;
}

$result = sort_again($jsonArray);
echo "sort_again() results:\n";
print_r($result);

$querystring = http_build_query($result);
echo "http_build_query() results:\n";
echo $querystring;

Output:

(Note: Due to a code formatting issue, the output actually has a line break after the http_build_query() results: part, but I couldn't seem to get the code block to work correctly when trying to show that in this post.)

Now, to the second part of your question, if you are using Python to do a similar action as ksort(), the below code example shows how this might be done using the sort_again() function in a Python context. Also, as far as I know, a similar option for http_build_query() in Python might be something like urllib.parse.urlencode(). I did notice, however, that the resulting query string from this method is a little different than the PHP http_build_query() function. More on this towards the end of this post.

import json
import pprint

from collections import OrderedDict
from urllib.parse import urlencode

obj = {
  "tran_ref": "TST2014900000688",
  "cart_id": "Sample Payment",
  "cart_description": "Sample Payment",
  "cart_currency": "AED",
  "cart_amount": "1",
  "customer_details": {
    "name": "John Smith",
    "email": "[email protected]",
    "phone": "9711111111111",
    "street1": "404, 11th st, void",
    "city": "Dubai",
    "state": "DU",
    "country": "AE",
    "ip": "127.0.0.1"
  },
  "payment_result": {
    "response_status": "A",
    "response_code": "831000",
    "response_message": "Authorised",
    "acquirer_message": "ACCEPT",
    "acquirer_rrn": "014910159369",
    "transaction_time": "2020-05-28T14:35:38+04:00"
  },
  "payment_info": {
    "card_type": "Credit",
    "card_scheme": "Visa",
    "payment_description": "4111 11## #### 1111"
  }
}

def sort_again(obj):
    sorted_obj = sorted(obj)
    # Note: if you are using Python 3.7 or higher, you should be able
    # to use a plain dictionary here instead of using an OrderedDict()
    # because the dict class can now remember insertion order.
    # ref: https://docs.python.org/3/library/collections.html
    #   #ordereddict-objects
    #d = OrderedDict()
    d = {}
    i = 0
    for key in sorted_obj:
        if isinstance(obj[key], dict):
            d[sorted_obj[i]] = sort_again(obj[key])
        else:
            d[sorted_obj[i]] = obj[key]
        i += 1
    return d

results = sort_again(obj)
print("sort_again() results:")
pprint.pprint(results)

encoded = urlencode(results)
print("urlencode() results:")
print(encoded)

Output:

sort_again() results:
{'cart_amount': '1',
 'cart_currency': 'AED',
 'cart_description': 'Sample Payment',
 'cart_id': 'Sample Payment',
 'customer_details': {'city': 'Dubai',
                      'country': 'AE',
                      'email': '[email protected]',
                      'ip': '127.0.0.1',
                      'name': 'John Smith',
                      'phone': '9711111111111',
                      'state': 'DU',
                      'street1': '404, 11th st, void'},
 'payment_info': {'card_scheme': 'Visa',
                  'card_type': 'Credit',
                  'payment_description': '4111 11## #### 1111'},
 'payment_result': {'acquirer_message': 'ACCEPT',
                    'acquirer_rrn': '014910159369',
                    'response_code': '831000',
                    'response_message': 'Authorised',
                    'response_status': 'A',
                    'transaction_time': '2020-05-28T14:35:38+04:00'},
 'tran_ref': 'TST2014900000688'}
urlencode() results:
cart_amount=1&cart_currency=AED&cart_description=Sample+Payment&cart_id=Sample+Payment&customer_details=%7B%27city%27%3A+%27Dubai%27%2C+%27country%27%3A+%27AE%27%2C+%27email%27%3A+%27jsmith%40gmail.com%27%2C+%27ip%27%3A+%27127.0.0.1%27%2C+%27name%27%3A+%27John+Smith%27%2C+%27phone%27%3A+%279711111111111%27%2C+%27state%27%3A+%27DU%27%2C+%27street1%27%3A+%27404%2C+11th+st%2C+void%27%7D&payment_info=%7B%27card_scheme%27%3A+%27Visa%27%2C+%27card_type%27%3A+%27Credit%27%2C+%27payment_description%27%3A+%274111+11%23%23+%23%23%23%23+1111%27%7D&payment_result=%7B%27acquirer_message%27%3A+%27ACCEPT%27%2C+%27acquirer_rrn%27%3A+%27014910159369%27%2C+%27response_code%27%3A+%27831000%27%2C+%27response_message%27%3A+%27Authorised%27%2C+%27response_status%27%3A+%27A%27%2C+%27transaction_time%27%3A+%272020-05-28T14%3A35%3A38%2B04%3A00%27%7D&tran_ref=TST2014900000688

Now, about the differences between the http_build_query() and urlencode() options, when running the outputted query strings from the PHP and Python examples above through an online tool, here are the decoded query strings:

PHP version output:

cart_amount=1&cart_currency=AED&cart_description=Sample+Payment&cart_id=Sample+Payment&customer_details[city]=Dubai&customer_details[country]=AE&customer_details[email][email protected]&customer_details[ip]=127.0.0.1&customer_details[name]=John+Smith&customer_details[phone]=9711111111111&customer_details[state]=DU&customer_details[street1]=404,+11th+st,+void&payment_info[card_scheme]=Visa&payment_info[card_type]=Credit&payment_info[payment_description]=4111+11##+####+1111&payment_result[acquirer_message]=ACCEPT&payment_result[acquirer_rrn]=014910159369&payment_result[response_code]=831000&payment_result[response_message]=Authorised&payment_result[response_status]=A&payment_result[transaction_time]=2020-05-28T14:35:38+04:00&tran_ref=TST2014900000688

Python version output:

cart_amount=1&cart_currency=AED&cart_description=Sample+Payment&cart_id=Sample+Payment&customer_details={'city':+'Dubai',+'country':+'AE',+'email':+'[email protected]',+'ip':+'127.0.0.1',+'name':+'John+Smith',+'phone':+'9711111111111',+'state':+'DU',+'street1':+'404,+11th+st,+void'}&payment_info={'card_scheme':+'Visa',+'card_type':+'Credit',+'payment_description':+'4111+11##+####+1111'}&payment_result={'acquirer_message':+'ACCEPT',+'acquirer_rrn':+'014910159369',+'response_code':+'831000',+'response_message':+'Authorised',+'response_status':+'A',+'transaction_time':+'2020-05-28T14:35:38+04:00'}&tran_ref=TST2014900000688

From this output, it looks to me like the differences in the query strings (notably the PHP [] versus Python {} items) might relate to how PHP often uses arrays to work with structured data while Python has the ability to also use things like dictionaries (as opposed to something like just lists). I don't know if this difference makes a difference for your use case(s), but wanted to point out that the generated output might be a little different even if the PHP function and the Python method used are similar.

Additionally:

If you don't want to have + signs in the query string output from the Python example, the following change can be made:

Change this line in the Python example above:

from urllib.parse import urlencode

To this:

from urllib.parse import quote, urlencode

And this line:

encoded = urlencode(results)

To this:

encoded = urlencode(results, quote_via=quote)

The query string output should then look like this:

cart_amount=1&cart_currency=AED&cart_description=Sample%20Payment&cart_id=Sample%20Payment&customer_details=%7B%27city%27%3A%20%27Dubai%27%2C%20%27country%27%3A%20%27AE%27%2C%20%27email%27%3A%20%27jsmith%40gmail.com%27%2C%20%27ip%27%3A%20%27127.0.0.1%27%2C%20%27name%27%3A%20%27John%20Smith%27%2C%20%27phone%27%3A%20%279711111111111%27%2C%20%27state%27%3A%20%27DU%27%2C%20%27street1%27%3A%20%27404%2C%2011th%20st%2C%20void%27%7D&payment_info=%7B%27card_scheme%27%3A%20%27Visa%27%2C%20%27card_type%27%3A%20%27Credit%27%2C%20%27payment_description%27%3A%20%274111%2011%23%23%20%23%23%23%23%201111%27%7D&payment_result=%7B%27acquirer_message%27%3A%20%27ACCEPT%27%2C%20%27acquirer_rrn%27%3A%20%27014910159369%27%2C%20%27response_code%27%3A%20%27831000%27%2C%20%27response_message%27%3A%20%27Authorised%27%2C%20%27response_status%27%3A%20%27A%27%2C%20%27transaction_time%27%3A%20%272020-05-28T14%3A35%3A38%2B04%3A00%27%7D&tran_ref=TST2014900000688

And when decoded, should look like this:

cart_amount=1&cart_currency=AED&cart_description=Sample Payment&cart_id=Sample Payment&customer_details={'city': 'Dubai', 'country': 'AE', 'email': '[email protected]', 'ip': '127.0.0.1', 'name': 'John Smith', 'phone': '9711111111111', 'state': 'DU', 'street1': '404, 11th st, void'}&payment_info={'card_scheme': 'Visa', 'card_type': 'Credit', 'payment_description': '4111 11## #### 1111'}&payment_result={'acquirer_message': 'ACCEPT', 'acquirer_rrn': '014910159369', 'response_code': '831000', 'response_message': 'Authorised', 'response_status': 'A', 'transaction_time': '2020-05-28T14:35:38+04:00'}&tran_ref=TST2014900000688