As a follow up to this question: Implementing WS-Security in Progress ABL, I'm continuing my struggle to implement WS-Security in Progress OpenEdge.
My problem:
On every request to a specific web service I generate a password digest based on:
The nonce, timestamp and digest are then added to the Soap header of the web service call.
This works fine most of the time but fails in about 5 out of a 100 requests (see more info below).
This is how I generate the digest:
PROCEDURE generatePassHashNonceClear:
/*------------------------------------------------------------------------------
Purpose:
Generates a password hash for WS-Security
General algorithm:
Digest = base64(sha1(Nonce + Timestamp + sha1(Pwd)))
------------------------------------------------------------------------------*/
DEFINE INPUT PARAMETER pcNonce AS CHARACTER NO-UNDO.
DEFINE INPUT PARAMETER pcCreated AS CHARACTER NO-UNDO.
DEFINE INPUT PARAMETER pcPassword AS CHARACTER NO-UNDO.
DEFINE OUTPUT PARAMETER pcHash AS CHARACTER NO-UNDO.
DEFINE VARIABLE mBytes AS MEMPTR NO-UNDO.
DEFINE VARIABLE mSHA1 AS MEMPTR NO-UNDO.
/*
Set size of mempointer, add 20 since we are adding the 20 byte
SHA1-DIGEST of the clear password in the end.
*/
SET-SIZE(mBytes) = LENGTH(pcNonce) + LENGTH(pcCreated) + 20.
/* Put the decoded nonce first */
PUT-STRING(mBytes, 1) = pcNonce.
/* Add create time */
PUT-STRING(mBytes, 1 + LENGTH(pcNonce)) = pcCreated.
/* Set SHA1 returns a 20 byte raw string. */
SET-SIZE(mSHA1) = 20.
mSHA1 = SHA1-DIGEST(pcPassword).
/* Add password, SHA1-digested (so we need to put bytes instead of a string */
PUT-BYTES(mBytes, 1 + LENGTH(pcNonce) + LENGTH(pcCreated)) = mSHA1.
/* Create out-data in B64-encoded format */
pcHash = STRING(BASE64-ENCODE(SHA1-DIGEST(mBytes))).
/* Clean up mempointers */
SET-SIZE(mBytes) = 0.
SET-SIZE(mSHA1) = 0.
END PROCEDURE.
And this is how the procedure is called:
DEFINE VARIABLE cPasswordClear AS CHARACTER NO-UNDO.
DEFINE VARIABLE dtZuluNow AS DATETIME NO-UNDO.
DEFINE VARIABLE cCreated AS CHARACTER NO-UNDO.
DEFINE VARIABLE cNonceB64 AS CHARACTER NO-UNDO.
DEFINE VARIABLE cNonce AS CHARACTER NO-UNDO.
DEFINE VARIABLE cPasswordDigest AS CHARACTER NO-UNDO.
/*
Get time in UTC/GMT/ZULU/Timezone 0 and store
it with 000 as milliseconds + Z for timezone Zulu
Nonce is a random generated string
*/
ASSIGN
dtZuluNow = DATETIME-TZ(NOW,0)
cCreated = STRING(dtZuluNow, "9999-99-99THH:MM:SS") + ":000Z"
cPasswordClear = "SECRET"
cNonceB64 = BASE64-ENCODE(GENERATE-RANDOM-KEY)
cNonce = STRING(BASE64-DECODE(cNonceB64)).
RUN generatePassHashNonceClear( cNonce, cCreated, cPasswordClear, OUTPUT cPasswordDigest).
What I know:
This works fine in something like 9 500 out of 10 000 requests. But there's a 5% fail rate. Unfortunately the error message isn't helpful so all I really can see is that the login failed. The web service provider states that the logins are rejected because of incorrect digests.
What I did:
To test my digest procedure I created a small python program. This indeed creates different digests when I try it with the in data (nonce and timestamp) from the failed logins. I am however not a Python programmer so there might very well be something wrong in this program (but it would be a very strange coincidence that it also should work in the same 95% of all cases).
Here's the python program:
import hashlib
def createDigest(Nonce, Created, Password):
"This function returns a digest"
NonceB64 = Nonce.decode("base64","strict")
pdgst = hashlib.sha1()
pdgst.update(Password)
PasswordDgst = pdgst.digest()
FinalDgst = hashlib.sha1()
FinalDgst.update(NonceB64)
FinalDgst.update(Created)
FinalDgst.update(PasswordDgst)
FinalTxt = FinalDgst.digest().encode("base64","strict")
print "Final digest : " + FinalTxt
return
print "This digest is repeated in Progress OpenEdge"
createDigest("tGxF8+DAmJvQo93PNZt5Nw==", "2015-04-08T20:10:44:000Z", "SECRET")
print "This digest isn't repeated in Progress OpenEdge"
createDigest("XdcAW1TdTr+MLp4t0QkJ8g==", "2015-04-08T20:10:44:000Z", "SECRET")
My real password is of course not "SECRET" and this makes me believe that the error has to do with the nonce. Changing the password to "SECRET" made the digest different but the discrepancy between the Progress and Python digests still was there afterwards (the first example above generated similar digests before and after the change but the second did not).
I have an open case with Progress Support but they seem to struggle with this as much as I do.
I've tested this in OpenEdge 11.3.1 and 11.4 on RHEL and Windows 7 and the behavior stays the same.
Answering my own question for future references:
The problem was related to codepage conversion just as @TomBascom pointed out but the actual error was really earlier in the "chain" than the SHA-digestion.
cNonceB64 = BASE64-ENCODE(GENERATE-RANDOM-KEY)
cNonce = STRING(BASE64-DECODE(cNonceB64))
In the second line the value of cNonce gets destroyed whenever the key generated contains values mismatching between iso8859-1 and UTF-8.
The simple solution was to change the cNonce variable into a mempointer and then rewriting the procedure that generates the digest.
/* Optimistic, should really be based on current symmetric encryption algorithm */
SET-SIZE(mNonce) = 16.
ASSIGN
mNonce = GENERATE-RANDOM-KEY
cNonceB64 = BASE64-ENCODE(mNonce).
And then the new procedure for generating a password digest:
PROCEDURE generateDigest:
/*------------------------------------------------------------------------------
Purpose: Generates a password hash for WS-Security
Parameters: <none>
Notes:
------------------------------------------------------------------------------*/
DEFINE INPUT PARAMETER mNonce AS MEMPTR NO-UNDO.
DEFINE INPUT PARAMETER pcCreated AS CHARACTER NO-UNDO.
DEFINE INPUT PARAMETER pcPassword AS CHARACTER NO-UNDO.
DEFINE OUTPUT PARAMETER pcHash AS CHARACTER NO-UNDO.
DEFINE VARIABLE mBytes AS MEMPTR NO-UNDO.
DEFINE VARIABLE mSHA1 AS MEMPTR NO-UNDO.
/*
Set size of mempointer, add 20 since we are adding the 20 byte
SHA1-DIGEST of the clear password in the end.
*/
SET-SIZE(mBytes) = LENGTH(pcCreated) + 36. /* 16 + 20 = 36 */
/* Put the decoded nonce first */
PUT-BYTES(mBytes, 1) = mNonce.
/* Add create time */
PUT-STRING(mBytes, 17) = pcCreated. /* 16 + 1 = 17 */
/* Set SHA1 returns a 20 byte raw string. */
SET-SIZE(mSHA1) = 20.
mSHA1 = SHA1-DIGEST(pcPassword).
/* Add password, SHA1-digested (so we need to put bytes instead of a string */
PUT-BYTES(mBytes, 17 + LENGTH(pcCreated)) = mSHA1. /* 16 + 1 = 17 */
/* Create out-data in B64-encoded format */
pcHash = STRING(BASE64-ENCODE(SHA1-DIGEST(mBytes))).
/* Clean up mempointers */
SET-SIZE(mBytes) = 0.
SET-SIZE(mSHA1) = 0.
SET-SIZE(mNonce) = 0.
END PROCEDURE.