Thanks to ndkrempel the date portion of the hex has been deciphered. The time still remains to be decoded. However, with ndkrempel's work on the subject I've realized the entire time stamp includes an additional 7 bytes which I had previously ignored since my files normally contain timestamps with zero minutes or seconds. Here are some "complete" hex sequences that hopefully contain the remaining time data necessary for an accurate conversion:
C00AE440555555555555D53F555555555555D53FABAAAAAAAAAAEA3F0CF712000000 = 17.05.1208.00.001024
C00AE440ABAAAAAAAAAAE23F555555555555D53FABAAAAAAAAAAEA3F0CF71200E0C4 = 17.05.1214.00.001024
C00AE440610BB6600BB6E23F555555555555D53FABAAAAAAAAAAEA3F0CF7120058C5 = 17.05.1214.02.001024
C00AE440176CC1166CC1E23F555555555555D53FABAAAAAAAAAAEA3F0CF71200D0C5 = 17.05.1214.04.001024
C00AE440CDCCCCCCCCCCE23F555555555555D53FABAAAAAAAAAAEA3F0CF7120048C6 = 17.05.1214.06.001024
Additional hex codes with various times
C00AE440721CC7711CC7E93F555555555555D53FABAAAAAAAAAAEA3F0CF71200E00F01 = 17.05.1219.20.001024
C00AE440CDCCCCCCCCCCE93F555555555555D53FABAAAAAAAAAAEA3F0CF712001C1001 = 17.05.1219.21.001024
C00AE440287DD2277DD2E93F555555555555D53FABAAAAAAAAAAEA3F0CF71200581001 = 17.05.1219.22.001024
C00AE440DEDDDDDDDDDDE93F555555555555D53FABAAAAAAAAAAEA3F0CF71200D01001 = 17.05.1219.24.001024
C00AE440398EE3388EE3E93F555555555555D53FABAAAAAAAAAAEA3F0CF712000C1101 = 17.05.1219.25.001024
C00AE440943EE9933EE9E93F555555555555D53FABAAAAAAAAAAEA3F0CF71200481101 = 17.05.1219.26.001024
C00AE440EFEEEEEEEEEEE93F555555555555D53FABAAAAAAAAAAEA3F0CF71200841101 = 17.05.1219.27.001024
C00AE4404A9FF4499FF4E93F555555555555D53FABAAAAAAAAAAEA3F0CF71200C01101 = 17.05.1219.28.001024
C00AE440A54FFAA44FFAE93F555555555555D53FABAAAAAAAAAAEA3F0CF71200FC1101 = 17.05.1219.29.001024
C00AE440000000000000EA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200381201 = 17.05.1219.30.001024
E00AE4404A9FF4499FF4EA3F555555555555D53FABAAAAAAAAAAEA3F0CF712004C1C01 = 18.05.1220.13.001024
E00AE440D8822DD8822DEB3F555555555555D53FABAAAAAAAAAAEA3F0CF71200A41E01 = 18.05.1220.23.001024
E00AE440666666666666EB3F555555555555D53FABAAAAAAAAAAEA3F0CF71200FC2001 = 18.05.1220.33.001024
E00AE440F5499FF4499FEB3F555555555555D53FABAAAAAAAAAAEA3F0CF71200542301 = 18.05.1220.43.001024
E00AE440832DD8822DD8EB3F555555555555D53FABAAAAAAAAAAEA3F0CF71200AC2501 = 18.05.1220.53.001024
E00AE440111111111111EC3F555555555555D53FABAAAAAAAAAAEA3F0CF71200042801 = 18.05.1221.03.001024
E00AE4409FF4499FF449EC3F555555555555D53FABAAAAAAAAAAEA3F0CF712005C2A01 = 18.05.1221.13.001024
E00AE4402ED8822DD882EC3F555555555555D53FABAAAAAAAAAAEA3F0CF71200B42C01 = 18.05.1221.23.001024
E00AE440BCBBBBBBBBBBEC3F555555555555D53FABAAAAAAAAAAEA3F0CF712000C2F01 = 18.05.1221.33.001024
E00AE440176CC1166CC1EC3F555555555555D53FABAAAAAAAAAAEA3F0CF71200482F01 = 18.05.1221.34.001024
E00AE440E42DCDC10BABDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200A28C00 = 18.05.1210.00.021024
E00AE4401DB1EFD86CABDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200A48C00 = 18.05.1210.00.041024
E00AE440563412F0CDABDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200A68C00 = 18.05.1210.00.061024
E00AE44090B734072FACDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200A88C00 = 18.05.1210.00.081024
E00AE440C93A571E90ACDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200AA8C00 = 18.05.1210.00.101024
E00AE44002BE7935F1ACDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200AC8C00 = 18.05.1210.00.121024
E00AE4403C419C4C52ADDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200AE8C00 = 18.05.1210.00.141024
E00AE44075C4BE63B3ADDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200B08C00 = 18.05.1210.00.161024
E00AE440AE47E17A14AEDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200B28C00 = 18.05.1210.00.181024
E00AE440E7CA039275AEDA3F555555555555D53FABAAAAAAAAAAEA3F0CF71200B48C00 = 18.05.1210.00.201024
Preface: I checked to the best of my ability and the hex I am dealing with does not seem to conform to the "standard" hex date/time formats (time_t, time_64, DATE, DOS date/time as calculated by Hex Workshop as well as all the formats checked by the DCode time decoder application).
I am attempting to write a batch conversation utility to deal with a propriety file format generated by an in-house undocumented piece of software. I have the ability to convert the files with a long forgotten and similarly poorly documented utility, but it is a time intensive and tedious process which I would like to automate.
Simply using a hex editor I've been able to find everything I need except for how the date and time is encoded in the original files. I'm confident that I've identified the 27 byte hex sequence that represents the data I'm after as I can edit it to change the date and insert it into another file to transfer the date. However, I want to understand how it is actually generated so I can covert files with unknown dates.
Below is a sample of the hex sequences (left) and the time stamps they result in the converted file (right in the format: DD.MM.YYHH.MM.SS). Some are taken directly from files generated by the software, others are sequences I've manually edited to see how the date is changed. Anything with a year >12 was the result of my modifying the hex. Does anyone have any suggestions? I am not much of a programmer and am over my head here.
80D8E340ABAAAAAAAAAAEE3F555555555555DD3FABAAAAAAAAAAEE = 11.04.1123.00.001024
E008E440555555555555D53F555555555555D53FABAAAAAAAAAAEA = 02.05.1208.00.001024
E001E440555555555555D53F555555555555D53FABAAAAAAAAAAEA = 07.03.1208.00.001024
600AE440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEE = 14.05.1209.00.001024
610AE440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEE = 14.05.1209.00.001024
6001E440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEE = 03.03.1209.00.001024
0101E440555555555555D53F555555555555D53FABAAAAAAAAAAEA = 29.02.1208.00.001024
01010140555555555555D53F555555555555D53FABAAAAAAAAAAEA = 01.01.0008.00.001024
01010101555555555555D53F555555555555D53FABAAAAAAAAAAEA = 30.12.9908.00.001024
600AE440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEE = 14.05.1209.00.001024
60FFE440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEE = 25.09.1709.00.001024
60FFE440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEA = 25.09.1709.00.001024
60FFE440000000000000D83FABAAAAAAAAAADA3FABFFFFFFFFFFEA = 25.09.1709.00.001024
60FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 25.09.1709.00.001024
01FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 22.09.1709.00.001024
02FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 22.09.1709.00.001024
10FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 22.09.1709.00.001024
20FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 23.09.1709.00.001024
30FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 23.09.1709.00.001024
40FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 24.09.1709.00.001024
50FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 24.09.1709.00.001024
60FFE440000000000000D83FAB0000000000DA3FAB0000000000EA = 25.09.1709.00.001024
600AE440000000000000D83FAB0000000000DA3FAB0000000000EA = 14.05.1209.00.001024
60FAE440000000000000D83FAB0000000000DA3FAB0000000000EA = 16.08.1709.00.001024
60FAE540000000000000D83FAB0000000000DA3FAB0000000000EA = 26.03.2309.00.001024
60FFE440FFFFFFFFFFFFD83FAB0000000000DA3FAB0000000000EA = 25.09.1709.22.301024
60FFE440010101010101D83FAB0000000000DA3FAB0000000000EA = 25.09.1709.00.051024
40DEE340ABAAAAAAAAAAE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 27.05.1117.00.001024
60DEE340ABAAAAAAAAAAE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 28.05.1117.00.001024
800AE440555555555555D53F555555555555D53FABAAAAAAAAAAEA = 15.05.1208.00.001024
Also, I should mention that the last 14 bytes do not seem particularly vital (although they do change in the original files). For example, the following sequences result in the same output after conversion. I don't know if those values are being rounded or simply stripped out during the conversion. Regardless, that part is not important if I can determine the month, day, year, hour, minute and second.
E008E440555555555555D53F555555555555FFFFFFFFFFFFFFFFFF = 02.05.1208.00.001024
E008E440555555555555D53FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF = 02.05.1208.00.001024
E008E440555555555555D53F000000000000000000000000000000 = 02.05.1208.00.001024
The following were generated solely from the source software and have not been manually modified.
40DEE340ABAAAAAAAAAAE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 27.05.1117.00.001024
60DEE340ABAAAAAAAAAAE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 28.05.1117.00.001024
80D8E340ABAAAAAAAAAAEE3F555555555555DD3FABAAAAAAAAAAEE = 11.04.1123.00.001024
E008E440555555555555D53F555555555555D53FABAAAAAAAAAAEA = 02.05.1208.00.001024
600AE440000000000000D83FABAAAAAAAAAADA3FABAAAAAAAAAAEE = 14.05.1209.00.001024
800AE440555555555555D53F555555555555D53FABAAAAAAAAAAEA = 15.05.1209.00.001024
The following are unmodified hex sequences from files with various times:
A0B7E340ABAAAAAAAAAAE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 22.07.1016.00.001024
C0B7E340777777777777E73FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 23.07.1017.36.001024
A0EFE340CDCCCCCCCCCCE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 13.10.1117.06.001024
80EFE340BCBBBBBBBBBBE63FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 12.10.1117.03.001024
C0EEE340ABAAAAAAAAAAE13FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 06.10.1113.15.001024
60D5E340ABAAAAAAAAAACA3FABAAAAAAAAAACA3FABAAAAAAAAAAE6 = 17.03.1105.00.001024
Any help or suggestions would be much appreciated.
I have made some progress decoding the date/time format. Specifically, I can read the date, but I haven't tried reading the time yet.
I believe the format contains lots of extraneous, possibly garbage, data; and only a few of the bytes have significance.
The date is encoded in bytes 0, 1, and 2 only (the first 3 bytes).
It's easiest to explain by interpreting the first 4 bytes as a little-endian double-word (4 byte) quantity x. Then the day number, as an offset from 1989-09-17, is given by
d = (x & 0x000FFFFF) >> 5
I.e. you only look at the 20 lower order bits, and out of those you ignore the 5 lowest order bits. You could summarise this by saying that the day number is encoded in bits 5-19 inclusive (in particular, it is a 15-bit quantity).
Probably the time is encoded similarly in a different position of the data, but there aren't as many examples of different times, and I need to sleep also!
Ok, many of the preexisting data you give (not the ones you've manually generated), seem to have tampered time values (conceivably the result of local time zone conversions by one of the involved programs). But just by going on the ones you generated, it seems the time is encoded as follows:
Take bytes 9 and 10 (by which I mean the 10th and 11th bytes), and treat them as a little-endian word x. Mask out the top 2 bits, leaving a 14-bit quantity:
t = x & 0x3FFF
This represents the time of day. It is in units such that 0 corresponds to 00:00:00 and (one past) the maximum value, 16384, corresponds to 24:00:00. This gives it a resolution of about 5.3s.
In other words,
s = t / 16384. * 24. * 60. * 60.
gives the time as a number of seconds past midnight.
Ok, looking at the extra bytes you've now provided, bytes 32 and 33 (the last 2 bytes) seem to be an alternative time stamp. Treating it as a little-endian word x again, it directly encodes the number of seconds from midnight. When it is 0, it seems the time stamp described above is used instead. It seems it can only describe times up to 18:22 before overflowing, so it might also use an extra bit from the byte before? There isn't enough data to say any more.
Byte 34 is a continuation of the above. I.e. take bytes 32, 33 and 34 as a little-endian 3-byte quantity, then, at least when it's non-zero, it gives the number of seconds from midnight as above.